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.■ . .. ' . . This report is Part I of a-stud-y-of program" ^ . , 
.ircplementation/ in J_2 models of Head Start Planned Variation.' Chapters 
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teachers as the data .source; „ (2) , t/ie , factors which may influence the 
extent to which models a'ir.e implemented,, such' as sponsor input, staff - 
reaction and . input, and the context in which implem.entation is 
undertakeTi; . (3) the corre.la:tions between the factprs w^ich explain 
vapiatio;as in ley ^ and (4) what the mode 

classrooms aire like in practice, using classroom iobservatio'n data . 
Included ..are a summary pif. findings, . . 

implementation studies, and data: tab^ ' .. . 
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PREFACE 



This report summarizes a^'study of program impie- 



, .mentation m the: second year of Head Start Planned Varia- 



tiort*. The primary questions tb be answered are r /t 

extent are the fexpetimental rtrGatm.ents actualiv . present • * 

in' the Head Start classrooms; ;arid what factors affect the 

■ / •* ' ' ■ ' ' i ■/*■*' r\ ' . ' ' ' ''^ ^ • •■ 

. succi5ss of : treatment implementation? The report'- riot only 
: .presents the results/ of our^ data analyses / but ;alspv make 
■ recommendations for f uture^z-iifiplementatiph/^^^^^ • The ; 

♦ bo<Jy of .the report is::r^cdntained in Part I ; ' Part II is a . 



volume of append icjes whi^ch present the details. 'df the; 
'analyses used in. Part I; copies'^bf data= col^ecticn- i;nstru- 
/ .msnts) and some additibnal. anal^ 

We aire. Indebted to the people at the Sltanfprd Research 
\ Institute who desi^^ and collected the^ 

iaata which formed they basi|s for this: report, v We' would also 
like to thank the : CCD staff / particular ly Thelma Zener - arid 
Lois-e^lliri' Datta, ior their ^a -'-^ ■ --l' 
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■ ' " overview ■ ■ ■'■ ■;■ 

, "^^if^-^:>^-n-^^^^^^ asan exper-' A- 

i-estig.te "the -impact of "various we 1.. ' 
. . denned ed^cationa,:- e^^^^^^ .y 

-'■ ^'l^ Head start child.".' ' , ': ' ,. . ■ . ' 

- . . - ■ ; school cufel^ia, ,;r :Sodei= a. they iili .eicalled ^ere. ik^J^ ' ^ ! 

: _ . :^:""*",°^--ti„g „ead;sta.t sites ,cr^^^^ 

• • . :»70-71,, there were 12 :.o^els and sites- in -planned ^Varla^ '^^' ' ' 

.. . . - ticn. .Por .15 sites m ^ ^dels, ,this „,as the-^econlyear ' of ^ -' 

■ , . - _ Partlcipat.;on,. for- the other sl.t,e.,;-it' was theVfirst vear " ' ' - 

. . T^l^ ^displays this information In ^ : 

' • ■ ■ ; ■ ,tv" °' '=*'^,i*"^l^e irt 1-9 70-71 jre 'V: ' 
P'^^-''^'^^1?5P^:'3 P?ogran« the n^ur V - ' 

■ - ;:.J;'^^^--^--^ >^ter.ined: Before 

V. "^'^^'^^"^"---"il^ o" educating- children, L^heciass'&^ 
^ v'. ■ - -^"^ -^^1= » a nu*er of dimensions; sueh:ai:ithe alount 

■' ' ■■ .Of structure in the classro.. or ■ thin^ortancg oV ac^lL e":^^^ / ^ 

.a^ies. The eleyenth^*aea, ^:th.;ina.lers; :ha.: a ■aiassr&ilt 
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Head Is tart Planned Variation Sites; 1969.-72 




1969^-70 



.19 70-71 



1971-72 



Pdr West 



Duluth 



Duluth 
Fresno 
Tacqma - 
Salt take 
Buffalo 



; Duluth 
■I sialt .Lake 
i Tacoma. 
Buffalo 



Arizona 



: , Oregph- 



Kansas 



LaFayette 
Lak6v;6od 



Bank / Street / Tuskegee ■ 
* Wilmin^gton 



\ lligh. S.Gppe 




Tupelo 

■E.V 'S^; - Louis' 



Graibi : • 
: Por tagevi 1 1 e. 



Ft. Walton 
Central 



LaFayette 
Lakewood 
Lincoln , ; 

Tuskegee. 

Wi lining tbn :; 

Boulder 

■v'Elfnirav 

Tupelo' ' 

. St. Louis^ 
E. -L^s' ;Vegas 

Oraibi; 
Portagfevilie 
Mourids . - - 

>t . Walton ; 
•eentral bz...i 



::;SeatVlet:v' 



LaFayette 
Lakewood 
Lincoln . 

Tuskegee :v 
>^ Iming t on 
^ Bipuider 

Eliti^ra - r ' 

„ Tupelo 
■.: E. -Las -Vegais 



Oraibiy: . " 
'Pbr tagevi lie 

■ Mounds ^-'V 

■ Ft. {;Walton^\;.:^^ 
. /.Central; ':pz\; • v 
V Greeley'^ : : ■ 

'Seattle 



- / 

■■■■'/:'■"■■' 
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«, Florida 



V Pittsburgh 

I^v/;.Enablers":..*^ 



V "Jackson viL^ 
/, C^att anbog - ^ ' 



.Washington 



Johnston Co. r 



^ Jacksonv i 1 le > 
Chattanb^^ 
Jon e s bo r d i - ^ 
Houston" ■ ' 



s Washington 

;y;Paterson^.:.;- 
: Johnston Qo. ; 

SLpcits.Hah^ 

' .Kansas .City 

/st. Thomas^ ' 

■ Billing s' 

Colorado' Sp. 
/Bellows Falls . 
'.Newburgh . 
, Puerto, Piico - ;. ; 



Jacksonville 
CHj|S^^riooga^,, 
JG\Qesboro':: 
^Hou^oh ; * . 

Washington :. 
Pate.r^pnv . , . ' 
Johnston Co. 

Lock- Haven ; 
; Monteyideb ' 



Kansas ;Gity . 1- • 
St. Thomas 

fBillings ;. . 
ColoradO 'Sp. „ ... " j . 
Bellows . Falis \ 
Newb urgh ^ " - * . t v ; . 
■ ::o" Rico % . 
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V.::- ^ pre -deve loped. • It, emphasizes the community 's 

curricula with the help of C 
■i , ^v:;:an early chil'dhood e^ who visits .-regularly . The twelfth- 



model in Planned ^^V^ the others in that 



A-' 



. .it;,,.i^ based, on the . premise that the parent is the primary v 
--7e(duc.ator ,6f . the child. Although all Planned Variation model^^^^ 
. include 'apparent component in ^ some way, this program/ i the 
•Florida m is actually a parent >• ' 

e'ducator . model and .:dpe s notbspecif y what should qo on Viri the" : 

■ -chi Id^ s: school/ eny ir onmentc.. ■ For^ . s imj>l ic i ty- of *^ presentation >^ 

■ we. /.of teiv^^^^ tbc,^.aii :.moaels - in terms' of . classroom . 

• y '< 0^ . . • . ■ 

:,i.,program5 ,,:,^recogriizi'ng' that ;'::this" is not appropriate' f or ^he 



Florida model. 



/ 

\ 



^-1 



' 1 



* ■ ; The " Use of vmodels dh vPlanned/Variation • is -asso^^^ 
with the .principle of^porisorship. The originator,'' 
: (or .sponsor) :of^.a is. responsible for" .deliver^ 

|; . : hi s progf an^^^^^ i 
only pro.vide manuals and materials , 'but also introdJce-T~* ' 
the model to the local- people, and provide training and^."' 
-continuing support' in-using^ it." . In this Report wevdis- o 
. -.tinguish 'between •model ' and ^ 

— — — - — — — ^T-"" ■■■ ■ ■ - 

' : ■ A brief description xTf the . planned Variation , models, is. . .v 
'"found in' a. seperate report entitled: , Smith, M. , "Some \ " 

-j^ ■■ . Short Tdrm Effects*, of "Project, Head Start^: A .Preliminary ' 
. : Report on. the Second Year of Planned Varrltion 1970,7r. . - 

■ ' ..71". The" liuf on Institute, JCariibridgp'', Massachusetts^. " 1973.- 



i). ■ 



...Jv 
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r ' ^ ^. ' . I}' 3 

■refer to the curriculum* or educational- program and : sponsor > 



; I to mean the person or people who deliver that model • • - / 

Each of ;.;the p Variation models works with one tov- 

> five sites. A site is a Head Start, center or „group of* centers 
usually located in the same community , and" admini'stere.dnb-y 
a Single : agency . Within: each site'^ the number of' classes ; > 

•) ' ' ' . , . • 

^. working- with a model^ varies r and in most cases it is less 

: than the total number of Classes in that srta. ■ In \14 sites, 

^ * * ' ' • . < 

the^plasses not working with the model are designat.ed as-: . Z — 

comparison classes whxch'arec to be used as controls ^in^^^^^^^^^^. V^ 

X .testing^, the ef feqts of the modelYtreatments . ' All Plann^ 

Variation -classes within a- site( work with the s.lkme model:. - 

The site is/ formally, responsible to the sponsor -for only the: . 

areas included in his model., The site is usualjly independent 

: m .administrative practices ahdvO^ 



\ 



Variation classes: \ • - ^ 

' ^" . ' , In. each E^lanned Variation class the staff is expected 

______ organize Its |jteaching : tov conform with model prescriptions 



/ 



•3'"'"^-'V- • - .-^ . .... ..... ............... .... 

~ Th^ere -IS; som^ ..qdesti^^ no_w,^as to what should -be included 
in ithe term^^/'modbl • .f". Originally , - it seemedL to' be limited to 
the;progra^^ from more current despriptldns 
it . appears that some ..mocJels'^ — ' EDC, for example — "also" 
consider their system'^bf advisors, and traininig procedure as 
J ipart pf jtheir models This;"fehanging_ definition 'will be 
.^J^iscUssed i^tef 'in^' t^^ chapter..- " . „ . _ „ ,i " 



are 'housed in ;the, \. 
.sam^ ,>ite's^;^^ Classens, there .are^^-^mr- 

^^^^jparrisron c^a communitfies. These ;:were^-^'lecte^d' . ' 

" v/hen a Planned Variation^site did not have'^enaugh, classes to 
f orm;a comparison group. ^They-^are intended, to' match the py sitles . 
^ ;bn ,relevant. d dimensions'-^ / ^ | 



/ 



or pririci|)les / • For ^ majority of teachers, aides, 
■'and volunteers r the model requirements differ from i-^heir 
us iial • mode s o f te ach i ng . . ■ I n--va r y in g- de g ree s ™f or-d i f f e rent 



teachers;V then y v/.orking v/ith a model requires tljat a Head 

':■ •■■ ■•.••■-..•*...,..,.,. , , . .. ,- .•:.-„ ..... ..^ ..... -,r . . ,S *. .... ■ . ... 

':S,tart -Staff -acquire new skills and techniques in .the class-, ,i- ' 
hew'ways of rei'atin^^g to children, and xn some, cases, new 
^values. ^ : , ; . 



IMRJiEMENTATION ISSUES 



'. \ ;!Th^'p.lanned; variation Aapproach is characterized by -its 
use of" experimental m^ This methodology, which 

can be jdescribed : in terms • of .conventions or Requirements,, 
enables, the researcher to create a situation': in; v/hich the 
effects 'of specified, variables /: or: treatments, ; are tested while 

->the operation" of other possibly: coit founding factors : is minimize 
The - advantage.* -of ; t experimental lapproach^ is Athat i t pi'ov^ 
the basis- vf of malcing inferences .about the. causeS' of the ' v 
"effects. -that are. "found. ' ' ' . ^ .* \' 



\ 



In this re|>6rt, we Will- not examine all requirements 1) 
• nc^'cessary" to a valid experiment , -but only two- related to 

tre,atments>. First/ the .treatments should -be well-specified; . 
^ In the case of HSPV, this ideally m.eans that models^' should be 
. ciearly defined'; v/e should have a. reasonably- detailed know-.. 

-ledge of what should \go on ,in a;cla'ssrodm durina the day . * 



with each, model. -It'' also means* ^ that the tre.atmcnts designated 



\v,. 



\ 



outcomes 



Second, the treat n>5nt s>. should " be fully ,XT:iDiQmxitM&' 

, 0 _ ; r*" ~ T" r— : — 7 — — 

If irnplementatibn. ist defiried-'^s th? extent to which' class- ' - 
^m.P.racti^G matcjie^moja^^^^ f uH . implementation - 



can be taken to mean that all. or most aspects ^pf the model, 
are pTresent. 



V 



These' requirements- are important for both the interna'!' - 
■^^ ; , , « ''^t " ^ ' 

•.and the exts^rnal validity of the , e^^eriment . Internal 

. validity,. .as Campbell and Stanley,, pu'i -it , is a -question 'of " 

(••Did in fact the -grxperimental 'treatments make a'.dif forence 



iri this : specif ic; experimental: ins tance? 



are' riot pres?>'^it, v;e cannot' an sv/er this question' af fiirmatively ' 

\ ' ' ' , *'♦.-/ ■ '-,/,-. i''" 

and as a- consequence, cannot\ interpret the results of the ^ 

study .\ ■ The ^reatme^t reqfuirements • can also >bG g-een as' issues 

of *exte,rnal validity. since a social e>cperiment is' intended ' 

. A 1 '■■ ' ,' ■ ' „' 

to Inform policy,/ it is. iiorportant to be alDle to' generalize' 



If 



a .01 f 

'I 

tine >t 



reatments' 



its -'findings. Bfacht and 



'A 



Glass state that one -aspect 'of 



external-^validity'is explicitly described treatments: 



i 



'^Generaliz^ation' and replication of^ the experimental results 
presuppose ^a complete knowledge of = all aspects of the treatment 

5 " ' ' , . . • - ; <? r_ 

Campbe.ll, S.Vand Stajiley, 'j. Experimental an'd Quasi- ' / J ' 
g^perime ntal\Designs for Res.ea rch-.-^vChicaao : Rand >lcNallv " " 
^.and Company, ^963^ p. 5. ' ■ , ^ , . 



The presenpe^ Af the treatments • is' hot ^the' only reouircment 
for intepal/ validity. Random selection and assignment of 
subjects; and the minimi-zing of- possibly confoundina variables • - 
are also i-mport\ant but are. not \ within'- the S(^ope-of this -report .■ 
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•an<a oxperiircntai setting. Adherence .t6' the -itreatiriont i\ " '/ , 



I . requirementb are necessar\* from s research perspective \ . 



because they provide the basis for a valid study and thereby ' 
enable xis to Interpret and generalize the stuti 's findings.. - . 



Meeting- Experin^onfcal Keg i^irei^^, n t s : 



,1 f ■ 



Aftej^ two years," however ;/lt''is'%pa>ent-that^ the. treat- 
ment roquireiments a^re'not met in Planned'. Variation 



The specificataon, of treatroeji ^'sr ' ,The/ req-uirement of 
\ specified treWtir/otits hs« ^rxic ii^ZJ^kK^^tt''"^ ' 



Specified tre'atiTcnts has two u^xinefel*ins.V~' Jthe- ryrogram iiAsdels""" ' ' S 

' • ^ " /V . - , '^%r'A : ' ^ • ^ - . 

should be clearly defined 4n'-c>per^fcx onM' •to tips V and 'the, • ^ „- ' 
treatments as designated, should include ail factors ivhich 
might affect outcomes:*:' .With re^-oect to the'ficgt dimension,- ; • 
the HSPV treatments^ , deviate for several'^reaSons^* Firsts not all 
sponsors felt that tlaeir models .wer^ fully .de;veloped^ at .the r ^ 
beginning pf the s^txidy.. Wi€h ^j:he exception^ of tv;b' models ^ - 1 ^ 
sponsors ^were'imablW to provide v/ritten\deicriotj ons *of what . * " 




J . expcrin:>ent/-'l-Jhi& feuch eyolu.tibn d's7eW9ationally. desif cbla ( 



If thB models' are inciompietg or, inappropriate, it cojiblicates" ^^-^^ 
- ^ the treatment from the researcher* s' point of view, Becau^eTlt^ ljfv>^ 



I. 

I 




I, ^Bracht , / and . Glass G:„'^e '^&xtirrtai:^.valXditr6£ <2xp&in»i^nti'r,e^S 
' . American.- Edxicational R esearch Journai\ J 1968/. 5/^ 4-37.-474^7^ 6;" 



treatments in themselves rather than reolicatibns of ' ^ 

the same treatKiont. . *^ ^ * * 

Second^ the requirement of clearlV--defined models is 
\ f ^ ^^-^ . - . 

not met because some nnoclels v;crc not complete and v;ell- 

defined even at the end of the study. During the third i>ear, 

.a^meifber of the Huron, Institute staff developed checklists 

int't^,nded to detail the comoonents of all ;r»bdels*® Though ^ 

refined through interactions* with the sponsors, many of 

these checklists are still only partial descriptions of a 

Class roojn ^program. This is a function of the state o^(^he 

art of educational theory, and does not result from oo'or- 

piannxng or a failure to take the experiment seriously • The 

models 3nc'liaded in HSPV\ reportifed^ly. were the-'bt^st available' 

approaches to earl;^ chjldhapjd educ^ation v;hen the study began. 

^ ' ^ " • ■ 

Pevelopirental theory^ however, does nat yet- provide an ade-* 
quai:6 basis for determininq v/hatv factors affect: a. child's 
grov/th and\ learning. Althbugh HSPV models .take Iheir focus 
from .tlxe 'different schools of ttought: in "the psycho] coy of 



'learning^;.,,.-^^' behavi^rist theory, Piagetian cognitive grov/th-, 
^i^'i^Q^fMY:^'^ learning, open education, total child development 
--^^ ifhese theories (v;ith the possible exception of thosc^ from'the 
behaviorist tradition) do not seem to be adequate yet for fully 



-specifying ant2 integrating an educational px^ogram. E>en if 
tHose theories were a^llecmately rerlet^ted in educational ;nodels, . 
'it is not.khovm v/heth(ir the^diffex-ences among models v;ould be 



differences v;hich are 



iinp^ortant to children's learning and 



%onaghah^ A t-^^'^Patterns of Iir.plerueniiatioTr-i^^ Start PlanrI.ed 
Variation;? The Huron Institu^e^ CairJ>ridge/ KaB-sochusctts , forth- 
coming ^ . ' . 



and . behavior. Further evidence, of the iinconplete state of-v- - 

N . ■ - ' \ 

this field: is -that the re is no .;,agr0ed .upon scheme Tffor ■ ■ 
.-classifying :the.- variation in the > approaches includea in this 
'experiment. It .is -not ^^clear that the models in HSPV repre- 
sent: a 'systematic variation of all relevant dijiiensions^C 
/ Third, some models are less easily 'dealt with i n an 
■experimental , si tu^^tion than others. We argued above 'that ^ 

X ' ■ ''".r I • " . 

soire models . do npt ; meet the criterion of a well-^def ined 
treatment because; of an inadequate theoretical base. - But ^ 

. even -if aM models V we r complete dn the educational' sense 
of having. a. >i:lear and integrated theory of how to - teach : 

-children, some, of the sponsors still v;ould not be £^ 

•• -. • ■'■■•< T ■-. ■ •.{).*. ■ ■ .■ . ■ * 

provide detailed, operational descriptions' of day-to-day 

\ 

..classrobm. activity because : their philosophy^ runs counter 
; tjKis^.^a , , -'^hese models , « which set ;out ' principles. '"cUtS • 

.encourage individual-^ adaptation in >imp-lemen-ting them, are > 
l^ss, amenable 

are" models whidh are defined in^'detailed operational terms/ 

We -conclude, then, that the models in HSPV are not 
-jexperimentally;.v/ellrdefined^^^^b^ ch'anaed- during 

.the course o^ the.tsxperiinerivt^^v^ v;hich 
they are bas/ed axe not always fully adequate for describing 
an educatioinal program, 'and because not all the models are 
defined in operational terms. ' ' ' . ■ 

A se^cond aspect of the experimental requirement t)iat 
treatments be v/ell-specif ied is -that a of .the 



treatment are identif ied •• In/HSPV/ previous de;^^^^ 



;of > the- treatments have limited to descript:ions, 'of ^^^^ 

..' v>...;" .'• . ••.'f ' ' y'y ' ■ : '. ' ■ ■ ■ 

cuiar . programs fpr children . In' practice, how^^ -thcj. treat- . 

v ment associated with a -'model is more ;than "this .V . 6n^. addifcioriai 

dimension IS training: a sponsor • is respongible^f or ih<;^"d\icing 

the program to the local staff and training th^m in its use. • . 

Anaiysiss ia C through 19) '•indicate that . 

thefef -are . systematic dif ferences amofig both models, and sites * ; ' 

' within mode Is^^^^^^^ the extent: and types' of training-^re.Gei-ved^"--^ 

by the teachers. Moreover r . anepdGtaJ.^eyidence indicates- f^'-'^^'^-^ 

•that\spons;i)rs oft^^^^^^^ a- broad range^ 

.which' cannot be subsumed under;:eitl:;erl,sta-f 1^"^^ the 7^^'^ 

"-priOjgr am- f or c hildren-rv^h one case / for' , example / a sponsor J 

" sent three:' Staff members tb a site, for three -idays .to hold a 

iidretreat dealing with staff problems stemming from a controversyv 

over the ethnicity and competency .of a newly appointed 

teacher I another &ite, a sponger was called on by the.; 

teachefe aides to assume the foie of . .a childva:dvocat . 

teacher struck a child. Th^se are important aspects of -^he 

^ ^'r ^ ' ^ ^ 

treatments which may affect outcomes and therefore r cannot 

/ " ^ * • • • V 

"be ignored. Their omission-^rom the model descriptions indi- 
cates that the' requirement of well-specified treatments is not 
met. : ^. : ^ ■ .* •, • ■ " ■ •> " ' "■■ ' 

yhus, from the experimental design point of view, the ^ 
treatments in HSPV, are not v/ell-specif ied/ either— in terms of' 

■" ■ • ■>■■ \ ■ / ..• • ■ ,.. ■• • •■ . :■ ■ •■ . ■ ■■ ' . . / ■ 

the-.curriculum models being well-developed or in terms of arirr"; 
relevant factors being included in the treatment description'. . 



ft,-., 
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. The impiieimentatioiit of "treatments : Prom the earlier } 
"definition of rmplementatibn. as the exte^^ which class- 

room practice, matches ••^model theory., ] a high level of , or full, . 

implemenj^45<t>^ can be^^t^^ all or most aspects 

' of the mpdfei .ar^ present. '^^^ A that full 

implementation; vbuld^be reached- by all models in HSPV, availr 
'"aSle. da^:a suggest 

"•Tables -^.^t;^, . and 6 (pp. 38/ 40f 44) in Chapter 2 show / 

the .analyses of our only direct source of information on 
.how.well t^he models are being amplemented: the sponsors' 



ratings of teachers' perfoirmari'ce in the models . The .findings> 
; ii) those analyses , of; variation in* the extent . of implementa-*; 
■tipn 'demonstrate that the trelatjneh^ not fully firesent. in 

all classes and that classes /within models cannoO^ 
. be taken as ;replications of a single treatment. • Moreover , ■ 
' the Teve 1 of Implementation /does not improve consistently 
royer time . as w^as expected at "the beginning 



These .analysfes are examihedr. in detail in C^iaipter 2. The 
;point to be made h^ire is . tHat the requirei^ent of fully 
implemented treatiiients is^ n^ / 



Thus 



s ; atter two years , ::it is apparerit that the treat- 



.mentg^^^^^^^ Planned Variation' deviate, from itonventional experi- i- 



^ V mental treatment requirements. Altho^dal/ the lack of treat- 
;. vnient specif ication and the failure to rfeach full implementa- j\ 
are ^^derstandable given the'^prac/tical constraints : 

■ ■ 'vV-/- ■■. ■ ■■■ -. : , ■■ . - "I ■■ ■ ■ ■ 

. - y, under. w|iich the:_s:tudy . hasj' been .operating - :e . g., , beginning /• ■ 
,..^ :without fully developed niodels/..^ 



9', 



'iB^ 



Se:e.> .for example , Offic^ of Cliild 'DeVelopinent > p.u 3 . 
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to- locations distant from the. sponsors;.. working .with bperating 
sites which face a variety of demands other than Planned^^^v -^^ '^^ . 
■ Variation -- they are problematic from a researbh poinV of^^.^^^ • 
, yiewi Because these deviations conf usis. bur- irit^^ 

causes of treatment effects, we, must deal with them if we. want to J 
. draw meaningful conclusions- from HS.PV. Our study of implementa- 
on, then, is an attempt to do this." ' - ■ ^ 

The Study of Implementation . ' 

: In,.the early -Stages; of^ Planned Variation, the study : of- ''''■'/<-[ 
. ■ implemen|ation w.as directed; ' toward ./two' ques tions : ; Are- the • ' 

'^^^y^^^^^^^ present , .and what are " thei mos t ■ e f fective . deliver-y • .• " 
systems - for : establishing the^^ m^^ sites.? Since . 

• it was assumed that .all models would ■ eventualiy reach full ■. 
>impiementatipn, these v^ere iseen^as. .essentially "straight- / 
'■'./forw^ardrissues^'^ ■ ' ^ \:v:'^-:- 

As. we have demonstrated, however,; it has become clear ' 
as Planned VariatgLon has progressed, that the notion of / " 
: implemehting comprehensive program ;.m^ 

Because of .the deviations from . experimental- requireihent sf ; - 



which result f the two original ■questi9ns' ■ 

must be replirasedr' : The' first shifts ^^^f .:, 
check.ing on the presence -of treatments £0 the question of -x^ .• 
what are . ;the treatments if" they a-re'-not the models as . originaKy' 
■ ^Qescribed? The ^ seconxl. ques tion Is not simply , an issue of • , 
/ ^describing delivery :s^ that- is ;stiTrliportant 

■ '^^■/^^^ "^^st" be broadened to- also-ask wliy is the^re - variatio n • - 
: v^ithih:- models^^ in the, extent ^ to which ^ the^ "are ' implemented? 



What are °° the treatments ?r The question ..n.f.^wha I the. ' ' 

/ -ft ■■ ' ''' I , ' '' ' • ' ■ : 

treatinen;ts are has -vtwo parts :|what iris aetually "happening 

vin^^•^e•Y Classroom^- ,:.;and;.-.to v;hatl extent does classrbom practice ' 
•ma^ch*model- theory? ■ f'l ■ • ■■ . ■ j ..■ ■ 

, . j The first 'parTv^thet"d"eterm happening ' ' 

in . the classroom, is important If or i: learning what "treatinents " 
the;,;Children are In fact receivingji This issue is explored '■ 
in-' Chapter 5 of this report usin'p^ 'data obtained with a 
classroom. observation ins t^irument developed by the Stanford 
-Research .Institute. This instrument is designed to be used 
with' all models and, therefore; provides geheral- information . 
Model-specific observation checklists were -developed by the 
Huron Institute for use during the 1971--72 school year. The 
data;., from these check-la;?3ts-are not; available for the 1970-71 
analysis, but are .presented' in a separate report for 1971-72 ."'•^ 

' Th^ questrph of what; is going; on m be directed 

to an .examination of . variai)les outside the classroom. We 
have 'suggested that the non-model activities "by the sponsor 
and other people may affect the treatment. Both of these 
...areas are considered— albeit inadequately -- in Chapter 3. ' 
■. ,: The second question, how: weil the models are implemented, 
involves a comparison, of "what is happening" with a standard. 
A crucial issue here is defining the ; standard —defining, 
f^li :inipleinentation. We all assumed that we knew what we \ ' 
meatgtV.in using this term, but no definition was ever made- 
explicit... In a conventional ^experiment we would define it! 

^See' Monaghan . •" ' -,' ■ ' • ' ■• ■ 



in terms of replication: we would expect the treatnient td 
be identical in all-' situations in which it was tes/ted.' m. 
Planned- Variation /..where we are working with, comprehensive ' 
treatment; ip "re^l li fe- situations this standard' seems". 
unrealistic.;. Thj, important factor in defining an alt^ernkive 
standard^J^^^ variation, .among, classes : .we should 

-accomodate the f actTh-iF^-medBl-^ 3p^^^_: 
sor and, the tea/chihg staff, to the ; unique situation ih.a.TitT 
pr even to^a class within a site, a^d to changes in a . •• ^ 
. .situation over. time. Even in .those i^o^^els .which do not ' 
•/emphasize adaptation., the glasses are no^t expected ' to' be " ■ . 

/ ""p^^ variation: 

-W^ must detery;ine how much variation^ can exist while .still 
considering a . class .to be a" fully: implemented program. We 
•must be ableji^.,.decide at .which "poini the>arf ation~ is so ^ V1 
great that ftill implementation becomes partial,.and the'. . ' 
point :at whifch partial implemenitation becoWs nothing - no 
evidence of/the model at all. As y^t.no one has explicitly 
set out the/ limits of variation, for all models, and thus '\ 
there is J specific operational definition of full imple^ 
mentation. p This not to. say that sponsors cannot identify 
a well-in,p|Lementec^ d'ass^^ on^y tha.t^they have ^hot^' systema;ized 
or/at- least communicated, their ' criteria. In, the absence of : 
agreemenfjbn an alternative definition . of full implementation,: 
we contir^ue in this ' reporTc^ to de'skno it in, terms of replica- 
tion of the model although 'we recognize' the inadequacy of ." " 
this; appijoach.' ■ • ■ , .. - ' • •• 



V . Other probleins arise in attentpting to measure the extent 
to which a model is implemehted. Examination of the splonsor 
ratings of ; implementation , to be discussed in Chapter 2^ . 

• illustrate; the difficulty^ for example / of compairihg levels 
of parti alv imp lemehtationu Comparison across models is ■ 

^diffiqul^: because the scales aire not anchored to any common . 
■standard and because models.; have different compoherits r.arid 

- are at different, levels of operationaiizatipn. ^ Asi a 'result> ^ 

%e do '^nqt know Whether a; r^ of , say ,^5 (on a 9-p6irit scale) 

'has .the same meaning in. different models. The problem of . 

.Qdmparis,^ exists to somfe extent within models^ . 

class which is rated 5 may. hot look - like another with the same. 



ant i\" 



rating because different parts : pf> tiie: model may. be present j.^^ 
each . When^^^^^ true ,- it is 'not clear that we have 

'repli~ca^^ f /T ' " 

^'■-siyt^ raise these issues in an aLttempt to demdnsfcxite ,:tha^^ 
.the question of how weir innovation" is; implemented is- 



xJiff icul^ as well a^ important^^ /This ;is the^ case not simply . 
because of inadequate -measures ^t also because of cdhceptual 
ambiguities . We' have not . resolved these dif ficulties irf. * : 
this report -but point them out ^br consideration ..in 
studies." ' / " .-■ ' V 

".-s Why is there^ variation? ' // . . ' ' \ ' . '-"^ 



• Bec£«ase - of vairiatiqn in |ley^^ we . ^ - 

are interested in identifvincT the factors which* affect / >^ 
-implementation not simply from the point, of view^of desf ribiri'g/ 



the -process , . but/' from the perspective of,. explaihinaV' • 
outcomes: ' v/e ^yant to explore .the question of why implement- 



ation is- more successful in one classroom than in another.. 

••• . v/. • ■! .• ■ ^ 

. .One group of variables which". no doubt , affects implements 

. ■ . ■ ; " ■ ' . 

V;tion . is spon'sor input. : We expect that this category include.«5 

j\ " • -' .. / ■ : . : • ,. . , . • A ' 

''"9^.0^^ .factors: such .a s.^, type 'and amount of training in the'' 

model , or/ feedback and 'continuing, support, but^^ i 
sponsor Vs staffing patterns, and- the^,^le^ he assumes. More 
importaW, we profiose that the process of implementation 
.depends^ on more ' than the . sponsors • delivery sy*s terns. imple- 
ment'^tion, is not determined by; sponsor input alone, but'° by. 
thp sponsor in .interaction with .the local staff working wi4h 
the model-, and with the context in which the Implementation • 

/takes place; W e contend that the /involved staff members are i 
not passive recipients of .the model: they, are not. equally 
capable and equally desirous of working" with th^"mpdlels . We. 
believe that implementation is -influenced by a staff member's 
reaction, to the model, to the sponsor's staff, and by his- or 
her past experience and • training. Moreover, we expect that 
sponsor and local staff inputs are influenced -by, the context i 

'in which, they interact. We propose that the si te~^dmin is taxa- 
tion; and the priorities of the administrators as well as^ . 
characteristics of ; the site,' such as its si'ze and location," 
affect implementation. These categories will*be presented 
in more detail in Chapter -3. ' . ' . I . 

• Inputs from people other. than the sponsor's staff may 



cjlsov influence injplernentation. ^ This^ issue is discussed ; 
: bpth -in^rela^^^ to the .sponsor' s input and within the franie- 
work of administrative priorities. We vpredict that inputs 
which affept the classroom program/ but do not reflect , the 
model, occur Jwhen the Planned Variation experiment^h^ low 
prioritXi. among an. ^administrator 's concerns - ' 



J 



Summary of Implementation^ Issues 



Traditionally, two.. treatment requirements are 



important. 



'If'- 



°to a valid experiment: the treatments should be well- 



specified and they should be fully implemented. In Planned 
Variation , the' findings- of incompletely specif ied'. treatments 
and variation in levels of implementatioij indica'te that th^ 



.Another assumption of the original ; conceptions'/ of Planned 
Variation' as an experiment was that the^comparison classes 



;^do not work with the model, r The comparison: cl^asses are 



internded to provide a basis vforinconclusions aWout model 



effectiveness - by showing how well similar chi^idren db 
without the model . : There is evidence / -howeve^rv 'that -p-. - 

' some sites , comparison classes> as vv^ell,;*^ Planned Variation 
classes / work Iwith the >model. ExampleiS ;of this^^^^^^^ ' 
are numerous . | One OGD consultant reports / fior example that--: 
when "asked if J:he $350 : allocated'^ for mter^ials^^ 
would only be for the 8 "classes in PV/ (they'' local sponsor 
representative) said it 's' physically inippssible and all classes 
-will use the materials v" And ^another consultant states that 
'^'the conscientious teachers are trying tO/Yget as^ much as 
possible of (the model); for .their children and themselves, 
v/hether th,ey are techrvLcally included in^ythe ^ special program ■ _ 
or not . 'Contamination ^ inv the program /IS ^^^g It is^ , ' 

clear that because of suchc contaminatiqn / -comparifion/ classes^^^ 
cannot provide appropriate tests fcfr model ef fetti^^eness in * . 
some sites. Moreover^v providing; re sour^ training for 

all classes in a :Site rather ° than for only Planned VariationV 
^^suggests that model inputs are being diluted. ^. 



requirements are not nietK Our study, of implementation is ' .,1 
art attempt to understand and to deal with these deyJ.a±ions .j>>-"' 
This 'means^ first/ that cannot simply check on the ' - ro^^ 
presence of the treatmients , i)Ut must ask what •the. treatments 
are# . - This question involves. bbt:h the determinat^^ion of what 
is happening and the comparison of 'Vhat is happen ing""^-...^^ 
with a standard. Important to the latter category are the 

-problems. of establishing: a definition of full* implementation ■ 
which allows for and yet sets limits on variation among 

.classes., and of comparing/ levels of partial implementation ^; 
yhe need to anchor rating scasSLes is relevant to both -issues . 
Although we do not resolve these concerns in this peport, - 

' they should be dealt with m future studies .. ^ 

The task of describing delivery systems similarly - ^. 
should be expanded to >include the more general issue of 

•identifying factors which influence - implementation. 'The^.;v ; 
goal of- this undertaking is to attempt to learn why the level 
of implementation is a cpmpiex proc not only 

■ . ■ ' -.3 . ■ ■ .-. ■ . . - ■ . ■ ■■ • ■ ■ . . . . , ■ . . . ■ . , , ■ 

; on the sponsor's input, but also, on the staff 's^ reaction, 
—and~ thereon text in which implementation occurs i I f this 
^proposal is pi'oven true then we can " conelud4_that_ th^ varia- 
tion in levels of iinplementatiqV.v^^ riot from .a poorly 
.planned experiment but from the nature of this type of study.' 
While; this- report con:^ains analyses directed to this point,.: 
: d3ta>re_ not -available for all factors we think important. 



DATA ANALYSES 

Data Sources : . .,"~'^::/^:' 

■ '-' I The data.^used in this report are. dxaW.'^^ a- number 
Cof --s^ource¥. Six vinstruments-^designedJI^and adniiniste,red\ by . > . ■...• ".^ , 

- the- -^^tanf ord ReseaTchVlnstitute are included ; in the ajialyses : . 

. Sponsor^vRatihgs- o -r ^ ^/^ [ 

■ Teacher ■^'Questionnaire. ,.. ^/ • ' r ''v. 

Teache^r AMe/^ ' 
. - Sponsor .• Ifnplementatioh.. Repeat y ^ . ' . ^ •• 

. : : Final Consultant Report - ' ■ 

Classroom dbservatidn^ Instrument^^ ^ . . [7" 
' ^ ' ' r - \ ^ . * /a ' 12 . ' ' > y 

■ Copies of these' ihstrumertts. 'are included in^^^^^ B. 'The first: 

three Instruments, above 'prbvide Information which .can be related , 
to individual classrooms ; While the Sponsor Implementation 
Report and the.- Final Consultant Report provide... data for^ the:, 
site as a-whole w The -Classroom Observation 'Instrument, is 
,th'e ;only source of data which, systematically deals with people 
and activities, within 'th^ classroom. In addition to the SRI' . 

- iris truments / we have obtained: anecdotal informati on abo'ut 
implementation^^^ f conversations, with, sponsors and from ■ 

the narratiYe repprts written by .OCD consultants . - The- OCD y . ^ 
cona%ltant-s are childhood education experts , who v;ere ■ 

Hired by .: ]^ Child Development to monitor model -^....^^^S^^-:-^ 

Tmpieme nytatTo by visiting. Planned ^Variation sites regularly ■ ■'■■-K^. 
The sanfe ^cbrisaltant alwavs visits the^same site, . ' ^ ^ ' 



/Although va. gre'at deal of data was collected through 



theie instruments ; not all iDf it 1 included in this 're— 



port. The. primary reason for 'excluding items v/as; their 



/ .^ror descriptions of the admnistratiori of these instruments, 
;/ th'e reader should: consult ^'Implementation of-Head Start: Planned-^^^^^^ 
^Vari ation Tes tingi and" Dat a ■ GolJ.ecti on E*f f ort , Final Report ; - ■ 
• . by the ; S 



lack of relevance to the implementation questions which have 
/ be€in raised *here o Another iinportant reason : for- excluding • ■ 
items was that somf questions wer-e-; p'oorly conceived, Con- - 
sequehtly aiiibiguous and ■cannot be^i at 
ted. For example, question 1-9 of the 'Teacher' QuestionrTai re ^ 

• asks teachersV:tb outlinV^ t/echniques.^ they would use when 
pupils have difficulty in-s,even types" of sltuations^v^^^ . 
question is open-ended. . The\si tuati^ms are' vague iind' diffi- 
cult to deal with in '.the. abstract. . Not' surprisingly ,(;t.la"e . 

• ^ns^Jrers- are also ,va^ue and varied, to spch^/ap e,^'tei>/t .that^^^ 
^i.t woui-d' be- extremely difficult to summarize md ih'te'rpr%- ' 

the ^ responses from approximately ^OO tc^achers; ''This ques- 
tion was not ihcliided in our analysis.- There are other ; 
ainbiguous itejnos which were 'included, however. Because* or 

/the lack of alternatives. For the most part, probleinsvV/i-t±' ■ 
individu^al items]; v/i 11 ' b,e dealt with as they are presented" 

\ri th'e^tbxtJ One exaitip.Ie'^ferves .to iilustrafce^ the point 
here: , all questions on t^aijing/i^ 

.-^refer"only to trainin.g in general and da not specifically 
ask about- training in -the ntdael. If , 'as /we. suspect/ traini'rVg 
other than model ' training occurs in Planned Variation site's, 
the failure to distinguish;; between .them%everely limits the' -' 

^^conclusions v;hi<^' can.be drawn 'in relation to implomen cation. 
Thus ,^ not ' all the data collected are included in.hthis report, 
^d there, are liinitations to some data, v/hi ch are included." 



■%-l:X. 
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Mpre^over/ in other /-areas v 



;h;\cj}. ,\s'e\ believe^, to ybe''^- i^m^prtant^^^^^ 1^%^ 
cfnalysis ^!xh'':ia^ -.repbrt^'^ "^^ 



Analysis Strategi es: ^ ^ , ' : --/J; ^ V ^'V 
\ " /^The major portion of the 



' , i nVo 1 ve's' ,e x ami ni n g-^a 1 a r ge 
' . ;\ . : or. single 'items taken fx*ont^ 
" ,.cases,/ 'the class : 
/ ' . ' ^ in' some i ns tancesv-tHei 
-\?Iv; '\ ' "variable , the ' means arid s tandard ' 
.i-'V^; give a piGturQ';'bf yti^ content 




<ivt - : , show no significant; 

" the- results 



.55."^ M.:^'. : "': ; v/.''v - tfie; ianaiy.ses >'r6i&^ui;re- '^a*'''hx^i^^^ 



Be separated from the o'ffects of: the site in interactioft 
with the model. As result, we must always speak of "sites 
, within models". , . . 

. - ■ ; ■ ■■ ■,■ • ■■ ■ ■•-.-.o , A,./,- ••■■,<;■ 

.' Second/ models and sites are both considevre'd to be 
fixed rather than random factors., Although models aire ' 

;unquestiopafoly fixed factors in this study,^ the nature of 
the site factor is not ;clear. Initially, sites were considered 
to' foe random* This is logical in view of the goal of assess- 
ing the effectiveness of models which implies that findings 
will be generalized beyond the Planned Variation sample. If 
sites are fixed, no generalizations . can be made and the study 
is better, conceived as a series of descriptive case studies 
than as ah experiment.' [There are jiroblems with sites as a 
random factor, however. First, si>4:es were not randomly se-r 
lected .and 'assigned to. models. Instead, the Head Start 

■programs „iji certain communities , ■ chosen on the basi^ of parti- 
cipation in Follo^/ Through, geographical location, and recom- 
mendations from a variety of ' sources / were given the choice 



A random factor is one in whi<;h the intervals or groups 
includiad in the experiment are taken to be -representative 
of a larger population;"" conclusions., drawn about a random 
factor ijn the experiment are generaiizable to the total > 
populat'ijOi;i,,frora .v.'hich it is -drawn. Subjects in an experi- 
ment -ds; [an ^example of a random factor, 'in contrast, a fixed 
factor li;s one from which conclusions are not generalizable. 
The lev?2is or divisions- of a fixed factor ' are" of interest-v 
in '%hei!i|s elves and- arW^^liot, considered to be sajnpies of a ' '^ 
larger population. -Treat'itsents .in psychological ^experiments 
are.-good ^examples of fixed, factors, ' ~ ' " ' 
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of working with the s'anie. rTiOdel with which the- Ifollow . " 

through program in the community v?as working/ or not 
working with a model at all. On the basis of selection, 
then, sites are not random; At the same ti mo, it could, 
be argued 'that the sites in Planned VaViation do represent ; 
the range of Head Start "centers found .In " the ^country , and 
on this basis can be considered random in spite of the non-- 
rcindom.;sele^tionv. A second, mo.re compelling argument against 
using sites as a random factor is a statistical one. In this 
study, when sites are .random, the F ratio for model effects [ 
is foripe'd by comparing .the mean square for models with th^ ' 
.meaji-square for- sites.. With on! v one to five sites per ^ 
model, hov/ever,; tiiere, is not enough statistical power in this 
■ratio to find significant model effects. -Tests on modi*, effects 
ithen , may be gverly conservative and wa may tend t® ove riook ' 
real differences (Type II error) . - V/ith sites as a fixed faS^ 
tor^. the classroom rather than the site is used to test model 
effects. The results in this case are probably too liberafV . 
sigriif leant .,ef-fects in^ recorded, for dif f erences 'which, ban 
really be /attributed to chance . On the basis of this "analytical 
difference, .we have chosen to use sites as a fixed factor be- 
cause our strategy is, to report as many effects as possible. 
It , must be reinembered , hov.'eveW, that thsse tests may report . - 
chance differences. It should also be recognised that^uthe " 
decision to! use sites as a fixed , factor v.'as. Dost hoc and. w^as • ]'■ 



made for statistical rather than experimental reason-s-..^. • 
The decision shoUld.not be interpreted as strong con- 
viction that sites are in fact, fixed factors . , 

• Tbe third factor which must, be considered in analyzing 
jphe implementation data is that we. must deal v/ith an unbal- 
anced desixgn/ ■ There are both ' unequal numbers of classes 
within site^. and unequal numbers - of si tes within models. 
This^is unimportant as...far as the basic s tatisti.cs , corr^- 
lations , -and regressions included in this^report are^ qon- 
cerned. .With ' the analysis of Variance , however., it muist 
be considejred. An^. unbalanced design is problematic because 
the unequal ceil^si^es cause the effects , to be confounded. 
As a result , .^e F ratios* are biased , an^ the results are . : 
difficult to interpret. The problem of unequal sites .v:i"t>.iR 
models has been dealt v;ith here by artificially creating- a bal- 



anced design / after the data. /'^as collected;,.. ^Aith-ough-thi-s - — 
excludes some sites and models from the "analysis , the loss 
of the data is "seen to be^less pf a problem than the confouhdin 
effects.. ' Table 2 shows the sites, included in balanced, de- 
signs for three sets of analysii^. The criteria for estab- 

■ ■ . ,.■.(..■ • . '■ - ■ ■ ■ . ■ "■ . . " - 

lishing each of these designs can be found in Appendix A. 
The design 'for the analysis of the Teacher and Aide Question^ 
naires will be ..referred to as ^the "standard design"\because 
•it include^s the greatest number of sites arid because -i^t is * 



used most often. 



-2S- 



. • •' ■ TABLE- 2 : 

Models and Sites to be Include,d in Implementation Ana^Iyses. „of .: 



■Teacher and Aide 



Model 



Site 



Sponsor. 
■ ^ Ratings 



Qulest 



lonn.aires 



On ly^ 



:PjV.~NPV 
Gompar 



Far V7est . 



Buffalo 
Duloath 
Fresno 
Salt~ Lake 
'Tacoma 



0 
0 



Arizona, 



'LaFayette 
Lak'ewood - 
■ Lincoln ■ 



Bank Street 



. BouMer . 
Tuskegee 
Wilmington 
Elmira. - 



0 
0 



Oregon : 



E. St . Louis 

Tupelo 

E. . Las Veaas 



0. 
0 
0 



Hxgh. 
-Scope- 



Ft, . Walton B.. / 

-€e hi:xa±~Oz'airk^' 



Greeley 
. Seatt-ie 



0 
0 



Florida 



Jacksc^nville 



Johesboiro 
Chattanobga 
Houston , 



0 

.0 
0 



EDC 



VJ^shirigtbn V^. 

.Paterson V . ' 
Johnston Go. 



■Pittsburgh 



Lock Haven' 



^REC 



Kansas Gitv 



K. Y.U, 



St> Thomas 



Enablers 



Bxllings; ./ ; 
■^Colorado Sp.: • 
Bellov;s- Falls 
Newburgh 
Puerto Rico 



0 
0 
0 



* # ; of Models 



#. of Sites 



18- 



0 

.0 



0 
0 
0 



0 
0 



1^ 



0-. 

0 

.0 



0^ 



4^ 



Kansas . 


■ Oraibi * ■ ■* ' ^, 










. Portageville ^^> Wy 


■11 








. Mounds ' ■ ' / • 




.0 , . 





.0 



0 i- 
0 I, 



0 
0 



0, 1 

of. 



0 
0 



0 

r 

0 

J ■ 

0 



7-, 



19.- 



6 ^ 



21 



12- 



■ *The models and si tcs in "this column, wi 1 1; 
"standard design. ". : ■ ; ; :\ 



be refer red to as the 



..The problem ;of/unequal numbers of .classes within sites 
is dealt with tby using, an approxima^te unbalanced analysis : 
. an unweighted ^^means analysis ; . The unweighted .means 
analysis- uses' the harmonic mean of the 'cell 'sizes to approx- 
imate equal h ' s*. A criterion for using , this analysis is 
thkt cell sizes be/approx:^mately the>same/^^ The results ^ 
can be* interpreted- as if cell sizes\wGre eaual and \ ' 
the- effects were independent. ^ . "\ \. 

Hence the ^ analysis of the imp lemeatati on. data. 'is .-^V 
first determined by the original design of the study where 
sites are neste.d within models . .The analysis is. further 
determined, by decisions';^ after the data was collected: 

.sites are considered' to be a fixed factor; "baianced designs 
are./ created where necessary by excluding so/ne sites from - the 
analyses; and. the analysis*: of unbalanced data (within si^ 
is approximated; by an ImWeighteql me^ans analysis* bf variance... 



Seqrle , -S . Linear Mode Is - New York : John .Wiley and Sons . 

. 19 71, p. .36 5. '^"■■'f' ■■■■ ■ : 

15.'7"-': ::■/:■.•■:■■.■■■■■■„:■, y--'l.:-r---') '\:l :^;:-^:.:- ^J: , 

An alternati'Ve. .to the unweighted means-':' analysis is the . 
least squares analysis . \ This is an exact f ather'- 1^ an 
approximate analysis of unbalanced; data^:, The ^ r " ' 

the least aquaries solution^/ however , are difficult, to in- 
terpret because the ef fe.Qts* are notTTrrTdei^^^^ Con- 
sequently the order in which; the :.f . actors ,* in this ^ 
sites and models , are: .entered, changes -the sums . of squares 
which in turn may change- the findings about the 'significance 
of - the effects- . Since we have no. hypothesis about order, :v;e 
will, rely primarily on 'the , unweighted means •ah-alysiS; as a 
good; approximatipn of » the unbalanced ^ d^^ - Some 

least squares •^♦.analyses^l, are used "i They will ■ 

be more fully discussed there. / . . 



. Anc.lyses> will be. fully described as they are; presented in . \ 
• this report *V . ■ " - ' 

ORGANIZATION OF THE-' REPORT ■ ■ ;. 

■ The body of the reports is organized within the frame- 
work , laid' out .in this introductory chapter. Chapter 2 • 
deals with ^th.^ question of . how well the models ' are imple- 
mented, using the sponsor ratings :of teachers as the^^data 
source. Chapter 3' explores the factors which, may- influence- • 
the. extent to which models "are implemented. The, yafiables 
arei' discussed both as -descriptors of whatr-is involved in 
working with a model and" as potential explanatory- of vari a- f' 

' ^\ ' ' ■ ■■■ ' -'^ ■ ■■ ■■, .■ ■ ■. ■ ' ■ ^ //'■-■ . ■■■ ■ ' . ■ ' 

txonsm- levels: of implementation. The chapter is divided . ■ 
into, three . groups of Variables : sponsor inputs staf f reac- 

. -tix)Jl_an.d_j.npaitiv.^an^d^ - 

undeartaken. Witi^in each of these sectionsyiwe first c^iscuss X^. 
the relevant .data f roniv the teacher consultant , • and--sponsorj . 
questionnaries , and -second, evkruate ' the d terms of 

identifying f act:6rs which ' account for variation in implemehta- 
tion. Chapter .4 presents the correlations between "the variables 
introduced in Chaipter .3 and enters the variables into regression 
equations, in an .attempt to explain levels of ■ implementation 

;■ Xsporisor ' s:..ratings' of \teachers)*. In Chapter. 5 , the question \, 

'of -what the model classes;^^ai^ev^^^^^^^ in practice is explored ; 

: through, the class ropm: observation data.,' Finally^ Chapter 6 . 

■ suirjuarizes -the^^ r • • " ^ ..'^ 



/ 

. •/ 
/ 



/ 
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Chapter 2 
' . LEVELS OF "iMPLEMENTATIdN 



• One of the primary, questiohs to which a discussion of imple- 
mentation -must be addressed is the ""extent to which ciassrobm / 

; practl'ice Wtch'es model theory or in other words; how, well a model 
is implemented. If all aspects, of a model are present in a, class 
in the manner prescribed/ then the model can b^ 
implemented. If some r. but not all/ aspects of the model . are 
i^resehty then ..the^ model is partially . Similarly, „if 

, no cpmponentst! are present/^ mod^l is not implemented." v ;^ 

between theory and practice will be* called 
"levels of ■implementation" . ..Thus / a high level* of implementation / 



for ex^pie, means that all or -most aspects^, of the model are 
■■present' , • - -B-- . - ' ■ > '--^^ . "^-v ' '■■"A 

The determination of- the levels of implementation in this * 
'Study is important for two reasons . Firsts the. requirement .of 
testing the effects of specif ied treatments, . or ih^ \ - , 



.■':No explicit definitions of implementation were given in any of 
the instruments to be used in this report, ^e have derived the 
definitions used here_£rom our knowledge of .what was assumed . 
about the^ nature of' the study and about how. the models would be 
implemented,. ^^^^'. W^ ^of the difficulty, of def ining ; 

"full" implementation/ particu?jarly in relation to allowing and 
yet , setting Idmits -on variation among- classrooms ." - These dif fi^ 
cial ties have not been' resolved and we will ulse the definition of 



full implementation as duplicating thfe model 
because it is the one used most coinmonly. " ^. 



here y however , • 



■'■A 



.if 
■ 'J 



^tliis case, well-d models is basic to the;, cdhr., 

ce^tion or^^Pl^ as an experiment. V Unle^ 

whether thel experimental classes do in fact approximate the: . . 
models, we' cahnoi^^ draw usdfiil conclusions abTout the effects pf: 
'those models. In this sense/ the question of levels of ' imple- 
mentation is one \ of checking for; the presence of the specified 
: treatanents .. Second , ■■ implementation is . a question of '.service . 



delivery :„ can pre-establ-ishedv.:pr^^^^ be - distributed and insti-r 

tuted ; under ..a wide range of conditions? In general-? tlie modela^ ? 
included, in. Planned Variation were 'developed and tested in con- 
trolled, closely supervised situations , usually laboratory 
schools^, In Planned Variation, however, the sponsors must ^ 
.esta&rlsh their models in the complex real world. : ri ■ 

Sponsors cannot supervise implementation as closely as when - 
developing the models, because the sites; are. of ten far from their 
home bases . Thus , ' the ^questionvO,f whether these models can be , 
"delivered" is a crucial oner- J ' — 

: measure of'^^levels of implementati<)n from 

the 1970-71 data -is the sponsors/ ratings of the teachers . -As 
part of the Planned Variation evaluation, : each sponsor was 
asked, to rate indj.vidually every head teacher with a Planned' 
Varratipn class on a 0--9 scale ("not acceptable" to "outstanding"!) 



A, sample rating , sheet is included in Appendix B. 



._ph the basis of how well he or she perfbinned in the sponsor •s 

model. The r;atihgs are impreasionistic measures^ in t^>iat 
—judgments are ba^^ the sponsor's personal conception of 

the total models rather than on the basis of explicit/, pre- ^ 
determined criteria. In most cases ^ the; ratings were actually 
: completed by th^ sponsor's field staffs rather than. by the 
sponsor himself. Th^s> within one model!/ ratings were often : 
done by morie^ than one person. The ratinigs were recorded' 'for - 
five points ia^^t^ Octoberv February^ prediction of ^^^^^^^ 
(actual) May,/ and prediction ^6f faj.1. The forms^ 

however^^-wi^^ out' only twice J The jf ir s t seti of ^a^Elngs— 

was completed, in February when the sponsors werfe asked to judge 
.the teachers' performance for: the previous October and the ! 
following May> as well as for Febr^ The sfecond set^ done o 

~i.ii--May, iriclnidBd~^^ as 

^.judgmetnts'^for-.-May.- . . ' . ■ 

. Although^this instru^ient appears to be straightforv;ard; . 
: there aTre a number of is su^ be considered- in 

interpreting the findings drawn from it. . Firstv the ratings only 
i focus (^n one .aspect ^ o model : , the .classroom program. . Many 

= t>? the models have other components ^ involvement 

and take-home tasks, which are nq^ reflected in this measure. 

jOnly one teacher per class was rated; aides were not dncluded. 
Jf . a; person t^ than one - class r she, was still b^^ 

once.; i lri our- ainaly^ the ratings * for^ 

each PV class a /teacher was assigns . 



With- the exception of the Gordon- model / however , ^ the classroom 



program seems to be the most ^important part of the model , so that , 
ratings- of the class can be /taken as a reasonable, if- not perfect, 
assessment of the model.. In this same vein, many models empha- ' 
size the role of aides as ^G^lassroom. teachers, so that \imple-, 



mentation must be viewed a's' the result; of a team effort, rathey 
than that pf the; head teacher alone. It is possible the head 

• / ■ •1! ■ : ' ■ ' ^ • 

teacher is- doing: ar.j excel lentv joby with .the mcde't,-but the aides ' 
are not, or vice-verisa . //; In either case ,^/a rating- of the head " ♦ 
teacher ..alone does not T^kdequately ^represent the extent ' to which . 
the mod[el .is being imt^iemented. I be argued,, however, 

that fn the majbrity^^l^ perfqrmance of the head , 

teacher is indicative df the- team contribution to model imple- 

.mentation. So again, we conclude that the sponsor ratings pro- 

^v^irde^a-rrear^^ 

implementation^ - / ' ' ; 

. A second factor: to be 'kept in mind' in .using the sponsor 
: ratings is , the ratings were made . One problem 
; IS: that not all sponsors completed the second set of ratings in 
May as Ijad been . planned.:: In fact, the dates pf the second ratings 
;vary-consic"erably, with some n until the 

following fall. This undpubtedly lowers the reliability of 
-the measure. A- second problem .stems, from the fact, that the * . 



Soma.xatings were received by SRt^as , early as JuneX'9i rv/hileH - 
others were . as late as October 



ratings were actually made pnly in February and May; the October 
ratings. are not independent judgments. The import'an'ce of con- 
si'derxng this procedure is , illustrated by Table "3 which shows 

• that apparent improvement in implementation between October and-- 
February is larger than the imprpvement betv;een February and 

"May. If this relationship/is' A?alid, it provides important 
information about the^process ''of implementation* ' We suspect / y'^, 
however, that the increase between October and February is an 
artifact of the^anstrument/ rather than 'an. accurate reflection of 

.changes in teacher perfoSnance-f-^- Since the October ratings wete 

■actually made in February/ it seems probable thcut they are influ- 
enced by the sponsor's knowledge of the teachers V performance in 
February: because he expects teacher performance to improve • 
over the year, and because he cannot remember the teachers ' , 

'actual performance in. .October, the- sponsor may unknowingly . ^ 

"i-^rrive \at~the'TO'c'tobe~r^ rairiTigT^b^^s^iln^^^ 

■-' ^ . ' " ^ . ' ' ■ ~ 

amount from the February ratihgv In .an^ attempt, to^^m^ . 

/' ' " - . , ■ ' 

decision about the nature* of r.'ttifese ratings , we examined the 

correlations. : be tween them . Since the elapsed time be tween • 

October and February is 4 months and that between February arid* 

May, 3 months we would ^ expect . the- Februarys-May correlation - to 

" ;r ^ ' " ' ^ o ^ ' ^ ' . ' - ^ 

lie, at least as large as the October-February correiation. On 

/the ,contrary,^ we find that the OctqberrFebruary correlation is 

.86, while the February-May correlation is ,65, This supports 



^Th'e ^.predictions /or May and the fall have been .excluded because . 
" they "are not, appropriate for- determining' actual/leyels of imple- - 

'mentation. - /j/ " ......... . --^ ...... ^.\. 




TABLE 3 



' Sponsor Ratings 
Changes'Hn Ratings Betv/een Points in Time 



Model 


Octr ■ 




Feb'. 




May 


Far West : 


.,3.7':; 


. (1.3), 


5 . 0 ; 


(0.8)V 


■ ■ 5 . 8-.,; 


Bank Street ' 


4.0 


'(0.8) 


4.8- 


(0.3) 


5.1 


Oregon ' 


4.6- 


(IvO) 


5. 6 


(-0 7) 


- 4 9 


High Scope 


4.1 


(0.8): 


4:9 


■ (OVl) 


5.-0 


Florida 


4.5 


(0.7) ■ 




(0.1) 


5.3 


Pittsburgh • 


3.7 . . 


- .(1.3) ... 




(-0.1) 


' 4.9 


REC 


4.2 


(2.8) 


7.0 


:(-o.5) . 


■ , 6.5 


Enablers • 


4.3 


(0.^) ■ 


5 . 2; " 


* (0.4) 


5.6 




• 4.1 


(1.2) 


, 5.3 


(~oa) ■ 


5.2 








./'' 







I Note: ^ The ninnbers in par^nthesies. give the size of the. 

'Change in ratings between- the tv;o-poijits in time*^ 
. : ' Negative ^.jiuiTibers^ indicate that , the"^ average rating. 

- J. ' ' - goes dov;n in th^t interval. ' . 




the argument that October ratings are largely -a . function" of • 
February ratings. Therefore, in the -remainder of the analyses, J '". 
we will include only the FSBruary.and May ratings; 

Third, it must be recognized that the ,spon«ors' ratings 
cannot be used for comparisons between models. On the one ^tjand,' 
impressionistic measures are good because they convey, tha sponsor 
global evaluation of how the model is goingv and; this can be seen 
as a valid judgment.' On-:'the other hand, this apjsroach is limited^ 
because there are no- shared standards for making the- ratings. t 
This is particularly; true in judging' different models: in order 
to make comparisons among models^ we would have ,to^ assume that 
sponsors were using the rating scale, in the same way • Since we 
cannot make this assumption, ,we cannot iise th'i^s instrument ' to • 

compare models as to their level- of ^implementation , in. Planned 

'.-..^ „ ^ , * , ^ ^ 

Variation • The lack^ of "shared^ or at least exolicit/- criiieria" ' 
. ■ ^ ^ ^ . ' , ^ . ^^•*' -V/.. . 'j ' 

•is also problematic in terms 'of interpretln:g the- r^Wlts' or' the 

ratings. In many cases, it is difficulty if not impossible, to ' 

d*etey?mine v;hat a sponsor means in making..ris ratings'^" For- ' ^ : 



■ Even within'teodel's there is some question of .the coiriparabiliCy' 
6f ,^iiTiplemehtation ratings, since mie -person rarely 'irated "air - 
sites' in- a' model. We ^ assume, however that r in. working -together 
and>?orking' with the same model, a sponsor 's "staff- vfill" have 
fairly -comparable standards f.or judging '^■iinpleffientation. '^^t .t:he',t. 
s^e time,, we must- .recognise that this is only an,;assuit)ip.tioj[i, 
and we expect that it may be more accurate for so^se models than 
for others. 



( 5 ' -"7 

J- " • 



\ 



\ 



example/ do not know ^whether- a sponsor has the sme standards 
for first an<3 second .year teachers / hor do we- know , how he inter- 
prets the category "average". 



i 

A final factor which must be considered in usitig the 



\ . instrument is> missing 'data.- This is not a proKlem of '^the ' ' 
instrument itself, but of the data which results fro;n it.. For ' • 
- two models {Kansas and aDC)-, -there ari no data- on this measure: " 

.these , sponsors refused to use the instrument.* iRor' two ad'ditioRal 

' • - ^ ... ^ ' 

\ models (Arizona and nw) ^ there is^only information from. the 

' ' ^ \ . , ' 

second rating fonaj'the first was not completed for* any of' 



the sponsors* sites;'' in the reisaining, models, some' data 'are,' 
missing because changes in 'individual staff members were^'not-^ 
accurately documented. As a result, the data Bxe verv confused 
;and cannot be used: Because or missing i3ata^ the analyses have 



7?- 





been done 




classes* 



Thus, in tJsmg the''^ sponsor ratings of teachers i'eamiisf»: - 
be aware that they focus^ only pn.the head teacher y- and /pirimarilyf 
, on the d.assroomi„ 'that only' fehie ratings tfoV February=^aan'dv Ms^. ' - . 

provide independent iudgments-? and' that the ratings ^cannot, bfe''/t ' ...ts? 
..used to.wSke comparisons among ipodeisl,<> -within the-" limits- of- 




In the remainder of this chapter, we examine the findings 
.from these "ratings. We first compare levels of implementation 
'in sites which have participated in P,lanned Variation for two 
y^ars with sites which have participated for one year* Second^ 
we discuss. the mean levels of implementation at each site* 
Thirds we examine the amount of variation . between levels of 
implementation within^ models .and over time, 

- EFFECTS OF NUMB E R OF YEhES OF ,PARTICIPa.TION OH I^4PLE^!ENTRTI0N 

^ Using-.the sponsor ratings of teachers^ the first questioa 
to be addressed is what- differences are there between, first and 
second year sitesyin their average level 'of implementation? 
Mthougif''r970^71^ was- th'S second year of the ^Planned Variation" 
experiment^ .for some sites in all models (as -well as some entire 
'i?K)dels), 'it .was the first year of participation^ The question 
of differences' between. first and second ^ear sxtes is important 

wdu^^ predict that a site which -had worked >^it:h 
a mod,el for two .years' ^i^:ould have a higher level @f implementation 
than a site which' had worK^^d- v/ith it for one year. If, this is 
true^ we.' must separate year 1 from year 2 sites in subsequent 
analyses* To. test our shypotheses we' performed .a repeated 
measur<ss unw^^ighted means analysis of variance on the ratings 
for^ S^ebruary .and Mayfor sites in models which haC^beon in 



' ' ' . , 7 • ■ " - . 

-the experiment, for two. years. ■ We did not include models as' a ' ■ 

variable , in this analysis because .there was an insufficient numbe 
of comparison points for all. but two models. Table 4. shows the 
•means for each year at each point in time and the results of the 
analysis . of variance . Although the" meaVis for sites with two- / 
years of participation in Planned Variation are slightly higher 
than those with one year at each point in time, the differences 
are not statistically significant (p = .178) , The interaction 
between year of participation and time of rating is also insig- 
nifir:antv .. We .cannot say that the differences at either time are 
due to anything other than, chance. Thusy sponsor ratings of ■ 
teashers cannot ;be used to distinguish between levels of impla^ 
laentat'ion during the first and second years .- This - does not 
.necessarily, mean-^ that - there 'are ..no .differences in- the 'ifirst. and--; 
^second'year 'of implementation. It is possible that differences 
exist but that the sponsor ratings are' not sensitive to theis. 
-If ^ sponsor has higher standards for teachers iii second year 
sites than in ,/irst year sites, a second year teacher could 
perform at a higher level than a first year teacher but receive 



Four models, REC, NttJ, Pittsburgh and the Enablers, were not' 
part, of the study during the first year and therefore, have no 
second year sites. Since we are interested in a comparison 
betv,'een years", there is no reason to include the sites from . 
tl>ese models in this analysis. ' For -.a list of' which sites were 
included- in this .analysis, see Appendix -A, Table!. .- • 



TABLE 4 



Sponsor Ratings 
Comparison. of First ytd Second Year Sites 



Means 



Years of Par-EriciDation; 



One 
Two 



teo . 



4.8 
5.3 



4.9 
5.3 



Source 



Analysis of Va'riemce 

. Mean . '■ 
df Scuare 



• Year of ?articipati"on 

Sxjie '■ ' : ' ^ 

+ Class within site ' 



'. ■■'■Rating-time: 

" Year X;::Rating~\tinie 

Sites: X'-Rating-iiine 
^ ei ass- :Xv Ratings -time. 



F-tas5 



1 

14. 
85 



-1 

.14- 
89 



7.6 80 

9.783' 

4.153 



0.047 
0.142 
1.100 
0 . 835 



1.849 
: 2.356** 
NOT TESTED 



056 
0.170 
1.318 
NOT TESTED 



.% of, ■■ 
Total Sum- 
of Scuares 



' 1. 27 
22.5 7 
61.18 



o:oi 

6.02- 
2 ; 55 
12. 30 



Total 



209 



2.891 



100.00 



Note: 



An imweighted means analys is or variance is -used' to aDD>-6xi- - 
mate a solution for tan equal ceil sizes. A balanced desion was 
created by selecting an .equal number of- first- and second" vea'r - 
sites trom the total sample. Models are hot included in this 
analysis because there are an. insuf ficienc number of comoar^ - 
son points to maintain a balanced design. Ratines ^=rom Loth 
February, and May are included as a repeated measure, , 

■5-: indicates the vf feet used in testing the oreceding e^^fects- 

ciass-v/itliin-cite is used to te.<jt both sites and models 
. .-. because sites- a.re.. considered ^ as a fixed -factor. - 
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the same, rating. If crlii^teria are relative 'to previous levels 



pf performance or' tO;J^i^^^^ expectation, then real differences 



I • ■ f//.^'/ ■ / ' ' - ' ' '■'''/ 

in impleinentation/it^ay not be revealed. Regardless of whether 

v. -l ^- -^w : / .• ^ ■ h'. ■ 

real differences^'^o or dc^ not exist./ they are -not -reflected in 
this meaadre...ri^;' Therefore, year of participation will, be 

. , / 'M ■ ■ / • • - ■. ■ . ! - ■ / ■ 

^ignored/ in •':fy 

■ - -T . . . r.-v///' . / ' ■ ■ • "I - • ^ . ^ ' ' ■ 

MEAN/L§^LS OF IMPLEMENTATION . j " 

le second question- .to..be_. examined! t the/sp.onsor 
ratings of teachers is : to what extent does the- Planned Varia- 
tionvteacher' s performance match- the, sponspr' s conception of- 
his model ? Table 5 .presents thei means ajld^ standard deviatlons- 
f or sites ^and models. ' The; primary conclusion to be drawn 
: from .this table is that in May all models iare considered to be 



moderately- well-^implementedv but none are 



implemented. The' mean- ratings, for ^all models range from 3 . 7 



outstandingly well- 



^it 'is possible that- trie site level is too! gross to. reveal the 
ef fects vof the niimber of years ■ of participation on in)piementa- 
tion becaiuse there are' first year /;teachers| even at/ second year *, 
sices.. To- t-est. the hypothesis ■tha:'t- teachers who have been 
working v;i th the model 'for two years have^ higher implementation 
rating"s;, thar^ teachers ;who "-Itave been wc)rking with:^the model for ■■ 
one year ^ we performed a .repea*t:ed^^^ m means, 
analysis , Using only second year ' sites . (the^ is no variation 
in first year;:-sites) , - v/e found no; differences between first ai;d 
second year teachers in levels* of implementation. See Appendix. 
C Table.. 1, - " \ ^' ' . 
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, ■ ■■' -r ,TABLE,^5 

LEVEL OF IMPLEMENTATION: , BY SITE AND .BY MODEL 
" • . i 'mean . AND STANDARD DEVIATIONS 





/-■^ 








..May 












SD. 






SD ^ ■ 




Par West 


5. 


.0 


2 


.30 


■ 5., 


.8 


1 


.41'' ■ 


27 . 


Buffalo . 


i 




2 






, 5 


1 


.64 ■ 


6 . . 


Duluth 




,4 


2 






5 

1 mJ 


. 1 


CI 

. 


0 . 


■ 'Fresno- 


4, 


, 0 


i 


.41 











■■■ 4%-.-„i 


Salt I/ake 


6. 


,2 


■1 


.50 


4 , 


» 8 


• ■ 


.96 




Tacoma ; 


6. 


.8 


1 


.10 


6^ 






.84 




• 

Arizona 










5, 


► 1 


1 


.29 . • 


29 


Lafayette 










4 


7 


1 


36 


17 

x-v 


Lakewood 












/O 




.89 


6 


JLincoln 
















.03 . 


6 


*Bank Street . 


4. 


,8 


1 


.81 


5 • 


,1 


1 


.50 


: 32 


Boulder ' 


. 4. 


,5 




.08 . 


5. 


,5 


2 


.06 


' 4 ■ ■ 


Tuskegee . 


4. 


.0 


1 


.96 


4> 


, 7 


;i 


.03 . 


13' . 


Wilmington 


5 


5 




.93 


•J • 




1 


.19 


^ ■ 8 - , 


Elmira ■ / ' . 


5. 


4 


1 


.90 






2 


.37 




^Oregon' ' ' ' , : 


5. 


6 


• 1 


. 50. • 


4. 


9 J 


1 


-.91 


■ 16 - y . J 


E. St. Louis 




0 


i 


.63- 


5". 


0.: 


i 






; -Tupelp/ V ; 








.53.. 


5 . 


7 


2 


.52 


3 . 


■ E*v'Lai5 Veoas 


4 


8 


X. 




A 




2 


• DX 


■ ■ '/r " 






















Kansas • ' . No\ data 




















■ : : . 1 \ - •■ . 
High Scope- r \ >■ 


4 . 


9 


■ ■!' 


.39;. ■ 


.5. 


0 


1 


.37 


31 


ytv: Waltbn^ 


5. 


0 


.-I- 


;;5 3 - ; T 


4. 


4 








Central; 62'^ \ 


5. 


0 


i 


41 


5. 


2 


1 




XD 


•Greeley:: .\ -[''^ ' 


5 . 


2 




.96 


5. 


5 


1 


.00. . '^^ 


■ 4 


Seattle v 


: 4. 






.64 


4. 


7 ■ 




.52.;: : V 




Florida ^ 


5. 


2 


2 


.13 > 


5. 


3 


■ i 1 . 


Mr : 1 ■ 


■ 18 : 


.Jacksonville 


4-. 


0 


0 


.0 . 








. ■ ■ ' //'// ■ 




. Jonesborp 


5. 


0 


0 


.0 ; 




3 






3 ■ : 


> / Chattanooga , - 


6. 


4 




.53 


6 . 


2 




.83 - 


V • 9 ' 


.Houston 


2. 


8 


3 


.20 


3- 


2 




.75 . 
























Pittsburgh 


5. 


0. 




.58 


4. 


9 ' 


1. 


,34 ' 


7 




7. 


0 




.76 


6. 


5 




;54 ■ '\ 


8 . ■ 












3 . 


7 


1. 


,95 


11 


Enablers . 


5. 


2 


1, 


.53 ■ 


5. 


6 


1. 


,40 


29 



. , . , , , . 5. 

■ Colorado Springs 5.8 2.14 5.8 1.60 ■ . • _- 6 

.Bellows Falls - 6.2 1,17 6.2 1.60 ' 6 

Newburgh 4.8 ,1.39. 4.8 .1.39.,, ■ B 

Q Puerto Rico 4.6- .58' 6.0 1.56 4 
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be : These means and standard deviations are based on all available .d^t a-, 



' to 6.5 r indicating, that f 6r 9 , out of 10 models , the means f all .. 
within^ the category^ designated^ as "average" on the original 
rating form (ratings 4 , --S' '-a^rid 6" are included in the average 
category) . The tenth model. (NYU, 3.7) is only slightly below 
/this category. The problem with these relative ly straight- 
:J^orward findings, however, is that the category labelr are 
confusing:' the scale is not anchored. by an explicit definition 
of what constitutes an "outstanding" perfonnance> and the 
f meaning of "average" is not clear. Because of these ambiguities 
pur interpretation of the data must be cautious, it does seem 
fair to state that all sponsors seem to be moderately pleased 
with the implementation of their model on the average, but 
'that none are raving about unqualified succesF). 

. A second conclusion to be; d^^ from TalDle 5 is that there 
• is a.\grb3it ^deal of yariation within sites in; levels of imple- 
mentatipn; al is reflected in ■the large stand deviations . 



examination, of » the f requency, distrlbutio for these models 
;confirnis these findings/': T for May 

show that or: the ^ s 9 ou t; of 10^, mode Is ^ at . ]^^ 69^ of , the. 
teachers in each model are judged to be perfbririihg at average ^ 
or better levels; again/' the . tenth model is oniy slightly lower; 
with 64%. of the teachers being ratqd average or better . 



Examination of the frequency distributions for this ^^m^^^ 
show that in many si t^Sr the^ ratings ran^e from 3. to 7|6r 8. 

.; From Table 5 it>is also apparent that there^ is variation 
among sites in the same3 model in the mean ratings f^^ - 
: points in time. - In additionr there, are differences/between ^ 
'. the- February and the "Way ratings for both sites and models . 
Before we can draw iconclusions about the' importance of either 
of these observations/ however/ we must perform further ^ 
analyses to see if the differences are statistically s^ignificant 



VARIATION WITHIN MODELS / AND OVEIl TIftE ^ 

The major questions to te addressed in this 'section ^ \ 
then, are; how much variation in levels of implementation j 
exists within models r and how much ^change in ratings joccurs ^ 
between February and May ? To an swer the se que s t ions we r an 1 
a repeated measures unweighted means analysis of variance. 
The artalysis , takes the three relevant variables models^ sites 
withiir iTiod^^^^ and ratings times (February and May) ■ : 

inlx) account at the same^^^^ t . / 



l^It is/ important t^ • v V .: 
mod^is/and that as - a re'k^ 

separated f^ the effects of the site- in inter action with * \ 
the model ; -wis mus t al^^^ speak of "sites; wi thin . md^dels • " 



->' ■ 
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The "Sites and models incl^ differ 
from"; those in the "standard" balanced design d.escribed in 
Chaptte;r 1 because of missing data. In additionjto the models 
previously excluded from the standard design/ 



were excluded from this analysis because they did not complete 




the rajtings. The Arizond model was also excluded because "^/^ - 
teachers were only rated in May, making comparisons^ over time \. 
impossible., r The remaining six models Were run with three 
sites per model m order to generate a balanced design / . 
Table 6 shows the results of t^ ^ 

A number of conciusions can be drawn from this analysi-s^ 
The strongest finding is a ;significarit e;f feet for sites/ VOl) 
This .means that tKere arev; significant dififerences between sites 
withiri- the same model in the -ratings they; receive . Such a 



finding suggeats that spohsprs can .distinguish b^^ sites ' 

as to how well they are impleimenting the sponsor *s**model, and 
this in turn suggests that .models :do^ not represent uniform 
treatrtjerits in this experiment .^^ ^ ^ : 

Another- finding illustrated by Table 6 is £hat approxir- -' 
mately '5 8 % of the variance in ratings lies within sites . This 



An. analysis done for : -May a Arizona irieluded; does not 

al tei: the, conclu s ion^^. d r awn from the major analy^ 

- Table 2 .for a list of sites arid .models included in thiS' ' 



analysis . 
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TABLE 6 



Sponsor Ratings . 
Repeated Measure. Analysis o f Var i ance 



■I 



. Source 



'Model 

Si te Wi th i n Mode 1 
+ Class within Site 



Rating- time 

Model. ;X. Rating- time 
." •Site. X Rating- time 
■+^Gi ass X Rating-time 





Mean' ' 


■■.df-' 


Square 


5 ■ > 


3.540 


12 


10.385 


79 - 


.4.246 




1. 811 




2 . 490 


12 


■ •1.596. 


79, 


0 . 888 ' 



F-test 



' 0 . 834 
2.446** 

NOT TESTED 



2.041 ' 
■ 2. 806*, 
r 1.79 8 . 
NOT TESTED 



: % Of : •/ • 
Total Sum ; 
of Squares 



3: 05 ; 
21.44 
57.71 



0.31 
2.14 
3.29 

12.-06- 



Total 



19 3 



3.012 



100.00 



1.. 

I: 



\ 



Notes ; 



•■ .V ■■ •■: . . . 



Thel analysis is based on a balanced design with 6 jnodels v 
and 18 sites ; the balanced design • was created b"V e liminating 
models with only one site and by eliminating level I sites 
(roiriimal testing) Inhere necessary .^^^ . 
^analysis of variance is used to approximate- the solvation with 
at\ unequal number of classes within th^ sites . Since sites, 
are nested within models, the effects of a site cannot' be 
separated from th,e effects of- 'the site in interaction with - 
■the mode 1. .,.:„,:.. : ■■ ■.■■■^ .■ • ■ ■■ • 

+ : indicates the effect used in testing the preceding ". . 
i: '■. .ef fect;^ si^ ' * . 



* . 

• - 



P < 
P < 



.05 
.01 



indicates that- most )of the differences'; inMevels of imple-* 
mentation , are between Veachers^^^i^ q^ner£T/ 
all 'sities have both good, and bad teachers . This finding • 
supports, the assertioh- mao^e above that, expeririental /treatments . 
vary greatly across the classrooms -representing 'each model . ■ v- 

The third finding of .tn^,' analysis is a moderately 
nificant model by rating-time interaction (p < 05) . This' 
means that the rfelationship between Februa'ry and May ratings 
is dependent on the model to be consider:ed. ^ExamilnTati^^ of 
Table 7 shows mean ..ratings for the six models incTuded in the 
analysis. From. this table we can' slee that in- some models the 
ratings improve betw.een February . and May / while in others 
they decrease or remain essentially the same. ' The largest 
"change's "are^l'n™ Far^Wes t^^and "^Oregon": th'e^^ Far ""^^es t ^'^ ^ 



ratings increase by 1 . 1 ^points ^ and: the Oregon- ratings ^ 
decrease by > 7 r^^ Th^ interpretation of , the scores which 
improve or stay the' same is fairly clear r^^^^^t 

satisfied or more' satksfied^^^^ performanuie in 

his model in May as he was in February. The scores v;hich . 
decrease are more difficult to interpret. It i& possible that 
the change: m ratings indicates?^^an -actual- decrease in f .; 
level of i'mpleroentationy "perhaps^^^^^^ ' 
discouraged with the model at the end of. the year.s I-t is. also 



TABLE 7 



\ ' Sponsor Ratings 
Model Means from the Analysis of Va r i an ce 



Model 


Feb.. 




May 


Far V7est 


4 . 7 


(1.1) 


.-5.8 




0 


(0.1) 




Bank Street 


-■5.2^, 


5.3, 


Oregon 


■ ^ 5.7 


'{-0.6) 


5.1 


High Scope 


4 . 9. 


"(0.0) 


4.9 ' 


Florida • 


4.5 


(0.2) 


'4.^7 


En abler s 


5.3 


(0.5) 


5.S 


Total 

■ .... ., ... 


5.1^ 


' (0*2) 


, 5,3' 



Note: .The model .n^ans differ -from those "given 

Table 5 because, they are based .only' on data 
f roju :3 sites- in each model; Table"' 5 is "based 
orr!*air available data.. 

The^ numbers in parentheses indicate the; size^^^^^^^^ 

of 4i^he change ^in ratings between 
May: 



possiblev howe^^er I that^^^^^^^ decrease reflects a change in the;: : 
sponsor ■ s standards more than in performance ; . the sponsor may 
have higher expectations of implementation in- May .than in 
February^ such that performance that. was essentially the same 
at the two points in time would be given a lovter. rating in May, 
Tha information available does not enable us to . choose the 
correct interpretation V The absence of an overall effect for 
time of rating is ; somewhat surprising because the original 
assumption^ of the Planned Variation study was that all classes", 
move cbnsistently toward full implementation <K 

-, - Finally/, the analysis- reveals no significant/ dirfer ^ 
between- models* In .fact> . the F ratio for th^ effect is^ ., 
ejctremely small, {p > *50) • . Even empirically r sponsor ratings - 
cannot be used to determine whether one model i-s implemented . 
■ to a greater vextent than another * This rein forces the con-- 
ceptual argument expressed earlier against using these ratings 
for between-r sponsor coinparisons because the ratings do not 
necessarily have the "same meanings to different sponsors r Thus 
even if we wanted to use this measure to rank- sponsors,. v;e - 
could vHOt* There is no evidence" that the differences between 
models are due to anything other than chance* 
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SUMMARY 



■ .- • In using the. sporisor's rafc^^ of teachers as indicatdrs 
. of levels -of inipiemshtatibii-, . we h-ave, reached . a; number of con- 
clusion's. 



-Twentyeiqht .of the thirty-^one sites .for wIVich ' • 
there xs aata on this measure, on the average ,\are • 
reported by • the sponsors - to be .inoderately weli-\^ --. 
ijnplejnented;, but none are 6tttstand.ingly v/ei..£-» \ 



■ iitsplementedj. " ' ' ". ' '. • " . 

2. Contrary.- to bur expectations, .analvses reveal no' \ 

signxf leant differences- in levels 'Of implespntatibn . ' \ 

ObetweeS-^ firsthand second -year sites -and- hb.^verai^ ■ - '"i 

^ effect' of timaVof ratings, .There is a'-model bv'' ■ \ 

rai:-iiig;ti!r*e interaction which sh^bws- that xrog^^ 
- mentation- in son® '.models inip.roves beti%^eW'pebraary : ^-i ■ .■ ■ 
, - and May,.. 'A'hi'le. in bther motlels '.it -declines;' • ^lie.^ '. - ' - : ' ■ 
.conclusions diavm- front each of these . flndings/,must^-- 
:\ • be terstative. because of" a, lack oi:-knmiQ64& :6t-h6w' ' . •.■ 
the.rratings>\^re:- being' possible 'that, thg'- .: ' ■' ' 
findings.- 'ar^ .artifacts-; 'jsf ■ tlfie ifisferurmsnt ,. ' ritter-.' 'l ' '. -r'-^; 
. than.-; indications:' of actual . .levels ' ,.of • ifsplemgn tafcibnV---' ■■ ' -I. :;. 
3.. / Perhaps; ^mbre ii!iportani'> .. -f^ese,. ara vnb; signlficaiit.; ' .' ' ' : ' 
•;. ; "di;f fer^hcos b^fifee'eri'.-itjodel?,- 'in- ieVeis : p'f iirtpl^f^nta-^; ;1 

fc®s;.witiiin;.Uie-- mpdel^i^:, Mcii:m&y^^-^-mosCMf''i^^ 

■ tion in;-^his:nfeasura'if Jlthin sites. Altl^oa^h, as 



• •fr..- 



we have said, th'sre^ds a 'qus&t'ion about hcw; to, . , 
interpret the ratings,, the firsdina of -dftference.^" 
inaicates that sppiAors do distinguish between ' 
people and sifc.^s .witin diei'r model, judging s^l'bf 
them to be doing a/.better jo^ of iiTiplei^fsnfeing ^th'err ' ' 
'^'rtodel than.-others , ^ , 



■1" • 



4 



( Th-sse; findings^ then^ do suggest that iinplej^entafeion" 
is not pro.ceeding as had beeo' ekpected. 'fmey raise the \\ 
"possibility that ..implementation-, of a curricialar tnodei involves \ 
luore tnanr.a technical process of transferring a ^ifell-deve loped ' . ' 
n>odel.^ l^us/ the answer co the question rai.s'ed .at tlie beginning - ' 
of the chapter — - can models be delivered? — does "not appear ' ' ^ 
to be yes in all cases. Perhaps^ the sponsors cannot daplicate ' 
their jRodels ia^all classes,' 

The lar^^^. variation ia sponsor ratings -within' sites' - 
also indicates' that models do not represent a tmiform tre^t- . 
mkxitt as would' be ey.g©qted in an exper intent testing model' 
©ffecfciv-©nes,s. By ^siifiply. indicating with ^^ich ■ntodel a class ' 
or ^ site ^ working, then, we cannot describe ^the classrbosi ' 
program thte^re. U»huSf the" ""answer to the second question ' 
raised ^it- the .begiming ©f the__ chapter — whether the"tr^^t-, / 
me-nts 'are fully implimeht.ed-— appears fc^^ " 

_ ^hese possibilities, that iffipieiE-enta.tion is^noi^o--* 
,ceediW§_^-as e^^pectsd _ reqtair^ that , addict ionai qiiestionsie 



a^kod. • Mrstf\,if the tsmtmdnt. is" not rally, •ispiiisefitea/'thii 
"we mast ask which psriss of the model beiirig\impiaaictkt@a - r. 
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before can draw conclusions . about, ef feotiwness . • Th'is 

.g^esfeion,.is .d.e2^ with only, in 'a ' prelimiharY'tashim''in this: 
I'eport, but is' being studied, jtbre f uily . during ■■;the 19'71-72 I; 
yejar. Second, sinc«-modeis are not .belhg. implem©^^ in all 
classes/ w© roust ask why there is. 'v3'r.iation. / W»at factor s,' ' 
affect implemtetation such 'that all, classes'- are not; 'a^^^^ the 
same level?^^ The nesct two chapters • are devoted' to , this ques- 

■tiOR.;:- ■ ■ ■ ■ /:■;. ' '■ ' ' 



4 
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. Chapter 3 . . 
'■; ■ FACTORS VJ^MCH INFLUENCE IkpLEM£NTATI0^5 ' 

■ ' ' * ' ' ' .. 

■ ' ' - ■ - V ^ ■ ■ • ^ • . ■ 

■ . ■ In Chapter. if we assert-^.^d that the quest i/on of wrd.chr ■.■ 
factoYs aifecX'imple^^ can be interpreted in tv;o ways,, 

/ •. _ ,First^. /the' factors/ or variables ;r can be used, '.descriptively./- 
/ ■ ; . • ; In this context , ' the -relevant-- .i#^rnplententcjJ:lon q-uestions are , 
' ■ what does ,■ pariticipafcion- in -P ■Variation involve , for the 

... ftead;. Start staff and 'conununity^ and :how\does ; that, .participa-^ 
. , . ■ tiqn differ •fro-jn participatiotx' in <requlai i^^ 
■ : . . answer. '^th;e5e\questl'b^^^^ eKamine--.spacific ' . ; 

. ■ ^' aspects , of implementation.^-.. .such, as ■tlie., kinds' of trair.ing • - ■ 
■" received by tfe- teafehers.f . ap.d^ tbiH ■characteristics of the " 
■ ^ *■ te.a.che,f who are '.Working: -irj/yianjaea.'V in=-; .. - . 

terested' in , both comparing ■■experiences,' in . di'ffe rent .models • 
' "and •in.:'>:co?nparing- Plat?^n.ed..¥ariatior5vwith 

- . '• .vtiori* : ";.':The -dc^soriMive^ . fjTat^e^iork. ^ ks ' one iys'. which 'the ■ vbi,.:!- 
: - ■abl#s';;-.to,;-be: preset tad^ ^erej^originallv concaiv<^d* , ''^ ' . ■ 
, ,;Secq.nd, --th^r factors^,; which affect impla5fentatipiii:,.can 'be 
used to , explain lev&:l§- 'bE £!mlQ^^ last - /^ 

■ V .^.r ■ ;diapter-' we ■ £ound••^lar9e■: /Viariatid^ classes in. ■tt'^o.' extant ' . 

;/.■•.■;•• '•••'to >^iich ,they,,.^w^^^ .This leads to 

'V.,-the;..Quasti oxisbss ■'^<'hy is there a 

' ■ hii^e^-p^ class or site, than- in"' 



another? The same variables which are presented descrip- 
tively can also be used to . try to explain this variation. 
Prom"^hj:S perspective/ it appears that the process of . 
iituplementatibn is more complex than originally antJ'.cipated. 
VHiile we would clearly expect the sponsor's input to be an 
important determinant- of implement a tion'^ we think that""'this 
conception of impI<?mentation should beV expanded. Firsts we 
propose that an exaipination of sponsor ^inpuij should not be 
limited to training^ as. it has been pr evio-u.sly bi^'t should 
include ; other di^riensions^ siuch ^as staf fing patterns, and . 
involvement • Second^ and rt^ore iniportantly^ we propose that 
factors other than sponsor input influence ^mcdel implementa-" . 
•tion,- In. exploring these additional actor ^ -we-', have . broken 
".them into two general, categories: staff input and. the context 
In- which .the- staff operates* ' ^ ■ , 

■ The ifnportant of . the first category of variables ^ . ' 

rests.. on the ;assumptio.n that sponsor . input will" not be 
effective, in ac^x^eving implementation unless the model is 
: adopted and used bv\ the staff* The success of such adootion 

^Suppq^t for ;this definition is \found: in , the fact that data 
was coll ec, ted only on thlrS aspec.t;. of • the .sponsor ,^ s input . 



(or IzTfplementacion) depends both on hov; well teacher likes 
the model and is 'v/illing to make an effect to implement it/ 
and on the level of her skills and previous teaching methods. 

The second category, the operational context of the 
staff, is based on the assumption that the sponsor and staff 
inputs are influenced by the situation around them; More ' ' 
specifically, v;e; propose that implementation is affected by 
characteristics of a site (such as its size or funding . agency ) , 
by the efficiency with ^V'^hich a si,te is"mahaged, and by .the ' 
support which administrators give to a model. 

Thus,^' V/e see implementation- as - an„. interactive process 
which depends not only on ..the input' of the sponsor, but alf:o; 
on the reaction and input-bf the staff with whom the sponsor ■ 
is- v/crking- and- the context in v;hich they operate. 

; The data to, be presented .are taken from a.numbter of . 
sources: the Teacher and Md^ Questionnaires / the Sponsor 
:iiriplem*3n ration. RepprtsV the vFinal tons ult ant: Reports and . 
several atems from miscellaneous sources. Our primary analytic 
-tool is the unweighted means analysis of variance "as described 
in Chapter 1' .'(see p.. 26 ) . / Four se6s ; of analyses/were per- 
^formed, : The first,-, based on Planned, Variation classes jonly'; . 
examines di-j:fere.nces ' a^iong ' sites'" and models on specif ie.d 
variafoles-'^r^:: the; standard -bal ^The- second.xon----'''V' 

tras ts Planned , ,Vari at ionr-- wi th; nont-Planned - Variation : (PV-^KPV) ' 
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j and uses a ba-l-ancedVdesicjn with a smaller number of sites" ^ 
and models. The third and fourth , based on the sponsors • 
reports and -the consultants* reports?^, are simple site within 
model analyses . Since 'there is (fnly -one observation' per . 
site/ the site is :the. basic unit, and a balanced cJesign is 

. ■ .not necessary. , . ' ' 

One - analysis issue which has not been dp.scussed. pre"- 
; viously is the appropriateness of the analysis of variance ; /' 
strategy with dichotompus response variables . This is an 
important, consideration since a sizable proportion of the 

-liji^^' ■ 

variables which follow are dichotdmous . ' Although ,it may 
appear questionable to use an analysisv of . varian"ce in-* ^^^^ 
situation, Lunney v h$s recently, presented evidence that this; 
- departure from the usual assumptions of the ANOVA model may 
not be serious . A theoretically more apprppri ate ^ v^^ 
look at the cJata is a 2-v;ay contingency table (sites by ^ ■ 
presence of characteristics) . Suppose there' are s sites arid 



See Table 2 for the specific sites included in these designs. 

^■'3 ^- ' " . ■ ' ■ . -, : ' ': ■ ' s 

Sponsor reports were complete^ for only 20 sites ; There is 
no data for Tus con,' V?eiJcart or EDC. V Moreover , -they " were not 
done until atvle as t 2. or 3 months: after the schooV'V^^ar the. 
earliest report is dated Augus the- latest, llvce^^^ 21. : ■ 

For 9 -silies, 3 en ti re ^ models , thei reports were filled out on 

• Oct6ber')29 or later.. . .Consultant 'reports are availah)le .:for 34 ; 

:/ sites ■ ^ . ■ ■; , ■ 

^ Lunney, G. "Using analysis of variance: with a, dichotomous / 
dependent variable; an emperical study" . j/ of ' Educational ' 
Measurement, 19 70 , 7: 26 3-269 . ' :. ~ ' • ' 
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' m models ( . -As shown "by Cochran the total . with S-P degrees 
of = freedom can be partitioned into'' components with 
degrees: of; freedom for testing model differences and s-m" 
degrees of freedom for testing di fferences' among s;Ltes within. 

..models..' This alternative analysis was carried out as a * 

'check on the AN OVA.' The resultant findings show' no- serious, 
discrepancies from the: .findings obtained with the analyses'; '\ 
of vairiance model. ^ ^ ^he. analyses are summarized" in iTable 11 , . 
Appendix C/ The analyses ■ of variance are .incl^^ the 

. body .of the; report instead of the X ^..'s for simplicity of 
presentation, . / , ' / ' 

Three additional issues must be kept. in 'min^ 
these data: lach/of independence among variables^, = testing 

. large numbers of variables ,^ and using si tes as fixed factors . 
Firstr..the riumb^^rvof significant effects, is probably inflated 
since many of .the -variables included here are not inde--/ ^ > 

..pendent of each other both because . they measure dimensions- 
(such as behavior and characteristics) that are correlated/^ 
and: because they are structurally related: several variables 
may be taJcen f r6r|;^o^^ question. Second, testing large . • 



Cochran/ - "Some- njethods of strengthening the cominon x? 
tests.;" Biometrics , .1954 / 10 : 417-51, ' : 




.niunbers of variables, at the same time increases the • 
probability, of finding significant effects, by chance. Third, 
the use of sites as fixed factors , .results , in this analysis, 
in liberal F tests. A . t symbol ■ on analysesrOfrPV-only tables 
indicates model effects which would hav^- been insignificant 
if sites had been considered random . There are many of . • 
them. Each of these"^ issues leads , to' the conclusion that the 
significant"^ e ffects • presented in this chapter should be^ 
interpreted cautiously. We will liae. signi ficance levels ' 
heuristically , then i as rough es timates of di f ferences. and 
as indicators of interesting findings, but- not as .;the basis 
for grand inferences. ' <> * 

Hence, Chapter 3. is intended to examine a large number ; 
of variables relevant to model implementation in Planned 
Variation. The body of the chapter is diyided into three 
sections, corresponding -^o. this categories of factors presen.ted 
earlier: sponsor input^ staff reaction and input, and the 
operatipnal context of the staff. . Within each section , the 
variables first are presented and the. findings discussed.^ 
Then , " the variables are considered -'as ""pxedxctiors of levels 



Using, sites "as^ random .-factors results in overly conservative 
estimates of ; significance because of the small number of. sites 
within, each model. • / - , . v 
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of. implementation , and additional areas of study, are suggested. 
The second. Step* is more speculative than the first, but is nec- 
essary because the" variables with^which we- are working . / 
do >n6t adequately tap all dimensions which might affect 
model implementation. 

• Because there ' are more data specific .to the study avail-' ' 
able for sponsor input, the data presentation in , that section ■ 
is. much longer and more detailed than in the ^others. ♦ Conver-^ 

sely, the discussion of the variables as. predictors is fairly .. 

S,- / " .■ ■ " . . . • ■ ' ' ■ ■ [ ■ ' ■ . '' ■ ■ 

brief, both because there is less need to elaborate on the . 

anticipated relationships; between training' arid imple-- 

mentation since, these ■ relationships are more commonly'^ 

accepted than 'others we suggest ,- and because 'there : are ' . . 

fewer aspects of sponsor input for- which there are no data. 

In the sections on staff input and' on Gontext, the- primary 

emE)hasis . is placed .on exploring !'v; ays in which variables might . 

be used to explain variations, in levels of implementation-, 

and in suggesting additional variables which should be .r* " 

• ■ ■ ■ ■ ■, •■■ ■ ■ ■ ■ . ■ ■ V -.- - ' ■ ^' - ■ 

considered. The data presentation . in these sections is 

relatively, limited. 



. SPONSOR INP.UT • 

The sj^or^ responsibilitY in Planned' 

•Variation is to transfer the models ' from the sponsor's 

home base to the He^ad Start sites. Since the modes 

• ' ' • \ ■ \ . \ ■" . ■ ■ ■ / , . \ : 

of teaching required .by a model may, for many teachers, 

^d>mf fererit:\ f usual. modes , ■ implementation 

requires thcit«,a Head S.tart -staff member acquire new 
skills and techniques in the: class . and nevf ways of 'relating ■ 
to the children. The . sponsor bears .the responsibility for 
training the staff in these new models of teaching. In this 

•sjection we wild prese5it data on. training, from the' Teacher / 
Questionnaire -and -the. Sponsor Implementation Report/ Several 
aspects of training are, dealt with., . We, «wi 11 discuss data .- 
on the : length /^ f requency, types, and sources ; of both pre- 
service: arid in-service training, . Ih:addition, we will 
examine data on the • continuing support -and feedback provided;'^' 

"iDy the' sponsor;. \. '/ ■ , .- ' 
While, training may- constitute a : portion of the 
sponsor's input it does not define it completely. Therefore, 

•after presenting the available data, we v;i.ll discuss the 
th.e adequacy of the data wijt.hin the fraiuewbrk of the . 

•expanded conception of implementation presented in the - 
j.,ntroduc'hion,. and v;ill suggest other aspects of the .sponsor *s 
inputs£;>;hich should -be considered in attempting to deal with 



implemeritation more fully/ These include the sponsor's • 
staffing patterns and the non-m areas with which he 
becomes irivolyed in th^^" corumunity 1 ' \ . - 

Pre-^-^i^rvice Training ; , \ ' * ■ ■ - V 

Tables 8 > 9 a^id 10 summarize the results of the " ■ 

analyses carried out on variables relevant^ to pre-service 
training. Table 8 presents variables, for which there are ^ 
significant differerit^es / amohg^^ or among sites - 

within: models . In addition to displaying the signxficartce 
levels, thf=i table shows the partitioning of variance, A 
giving the pei-cent of variance explained by; a knowledce; 
of models and sx tesV and- the percent of ; variance remaining 
among ;, .cia?s-es^wi thin the sites .. The analyses are besed v. 
on the "standard c:esign'V" ( as specified in Table 2) and 
include only .'Planned ' Variation" classes . ^, / ... ' • 

•jTabae 9 presents.- results of the comparisons between. ;■ 
Planned, Variation and 'non-Planned Variation: classed (PV~NP\f) - 
. for the s ame variables th at wese analysed in Tab le 8 . Again , 
.both /sigmficance levels and the .partitioning of: variance are 
given; The , important;.ef rects ; in tJ^is j^table.; are, ^the' overall':'-; 



Ori.e-- geneiral .prpbiei^iawith : mos t , of - the.; ates.tidns' on vt raining . 
-they . arev not . speci.fi caliy. Iiniite:d:-t6; ■ / 
Using .■ them in; the /corvtGxt;V:of implgimentatibhv: then',^ ireaiiires ^ 
the -assuropti on;; .that: ,mos t , ■ ; i f- no;t-. kit -traihing ' -in.; a sit^', is . v;. 
related ;-to.:- the,..' model.; .. =SincY:'ti:'i.ere: is evidence;':-tiyat 
not always , the case." Ce> g. peopl^^ 

representative : are reported. - to give . trainihg) , - the ^frhai . 
presented here, duist be '.used.;- with ;:cautio^^ con-- :" . ■- ' 

elusions. abbut,::model i-inip:iementati:bn. " . :v ' ... . • ; :;: 
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TABLE 8 



Pre-Service T'rain,inq.' 





' J^--.-. ..Sources 


of 


Variance:- 








Variable l^ame 


■ ' i 

.1 

^5odei 




Site- 

Within^ • 
Model 


Class- 
Within' 
Site" 


df Class 




Did you receiver pre-- 

rvi ce (^s umme r)" -^^^ 
training? . 


18.15 










TQ#3 


How long was -tlie 
pre- service training 
period: # days 


16 . 16 


\ 


23.79 


'J. ' 


124 


TQi6 


. kind of pre -servi ce. 
-training: 

Demon£itra£ion lessons 


20. -.'31-; , 


12.76 


"66.93: 


135 .- 




Lectures- 


■ ^-3 ♦ 11 .'^ 

* it' if: - ; ■ 


25.65 

* ^ - ■ 


51.24 


135 




: Individual . meetings? 
with leader: / 


■ ** ■ 


21.-24 


,6 8.0Q 


135^ . 




Group: discussions 


-15.36 






62.13 


135- 




Discussion of video 
" •taped-^ lessons^ 


23.47 




13; 37- - 


6 3 . 16 ■ ■- 


^' - 135 




^Observations. ■ 


22 . 15 

- ■- 




13.35 


64.50 


135 




^- Role- playing.: .■■ 


18.80 




16.71 


64.49 


135 


TQI6 

■ r 


■^•Jh;^: gave ^-training . 
. (pe^;son Vno.-was .. 
check^^d one or. ii>ore.. 
• tinies)",\ . Sponsor: : 


18.91 

- 




11.5.2 


63.57 • 


135 




.-^Consu-ltant ■ - i 






13/.i.07 




135 




Local - ^ad Start 


14.18 

.- 




22- S3 . 


62.99 } 


135 




Other- 




T 


-14.56 


75.83' , 


135'. 



See- attached page "f.or no'tos* 



■Notes for Table - S.:. :' ' 

The letter and number code in tho iGft-hand column ' ■ 
indicates the guest:ionr4aiee and item ifrosn v;hi,ch tho 
variable is taken?' TQ is the Teacher QtJesti-Quilaire. 

"The top row of numbers for each variable qivos cne 
percent of variance explained by rsodels and sikcr ".-snd 
the percentage found v/ithin--- sites. The; second "row -of - 
numbers for r ch--variabie indicates , the '.significance ' 
■p-f" the. differences between irsodels' and sites. Con- ~ ' ' 
ventional. notation is used: ' ' 

•' p < .001 • " . 

'**■ =^ p <^ .01 • • 

= ^ -NS'^f not ~ significant 

Df class refers to tho-^degrees of freedom for clJss&s ' ' 
within sites (with t;\fc Teacher Questionnaire data-, 

class is equivalent- to ''teach<gr"^^ there i.s o-ne tcach« 
ex per 'class ?=yith Teacher Mde data, ^' class-" is eauiva- 
lent to "aide."). Since . there are always 18 df for " 
_sitss and S^df for stiodeis iri -thfe/' analyses shown in this" 
table ^ the information is -not repeated in che bodv of 
the table. ,~ ' ■ 

Sites are considered as. fixed faccorsi therefore) th*^ 
mear. -square for both models and sites is .tested aaalrlst 
xhe mean square- for classes within -si >t©s. • A ^ svmbol 
.'£ndicaces_ that model ^ef feces Wculd be insignificant if 
iRodels v/er-f tested againsc sites.. Tfeefe is no site by. 
Jftodel interaction term, because lites.. are nes-ted 'withia • 
models.. ^ • ' . 

The' analyses in this table are based on- the standard ' 
design with nin^s- it!o'dels\and 21 sites, Ip should be «oted 
that the an&lyses .in Tables 9 and 10 are based' on a di £« 
-terent set of sires and^TOodels.- ■ ' ' ' ' ■ ' , 

-The analyses of quest ions 5 and ,.6 are based jsn responses - 
rrom all ^teaoKeife, .including those v;ho rc-»or&ed-- n©' ore- . 
service training (question 4)» ' , " • 
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The top o^^nmb^rs '£6r aach'' variafolo~ aivv*s\ t3i>i 
cent- .of A^axiance explained b^^ PV-J/spv^" 'modoljsi/ ^Jitcrs , ^ lii^tfVfe.T;*. 
.act-ions crnd classes, wifehlil sites^^- /?ha second row for^'d\di:'/.'" 
Variable indicat^^s- t"he -.sigpiflcanci^ of; t.Hi^ eLfecr.s« "Convan^- - 

■ ■ ^, . ^^p, cai*.'. ^ / • ■ •;. • 



^•©s ■ ,are unweigh tea /iTie/ans ■ a.naiysis;'.,,ol:'-' variance 



' ; based /qn- ' a, balanced aesigri'..fV3: tft:-i6^' 'jab^els ' and! l2.;-sii:^s'. :'"'"ai1.. 
■' -©fBscfes- are.;. tasted ag,aiftst'^-clas^\>,;?ithi^^^ ClasB'-" 
. re£e.rs -fc{?:. the. /degrees of, freedom,- for ■'c.l,ass«3s; ;br;testch'ers>"'- ■' 
■wlchin s'ltfs'c '.The 'dog^sjess-'-df.,- £reeddmVf6K\->h©v;©,th'©^^^ 
_ , • are- ■'const snf- and are- therefo-re not-inpl^uded-in ' tiW. .■bpdy. 
tJie table'? ?ii6d.elS'. ha'-^^e \5 dt in' tMiB\s.h-&ixf$i&',' sPcqs .h&v^'-^ 'r 

.o- by. .cifiss,, have..6... , , ^ , '' ' ' 

:tofeai'- ana.%'Sf s o ^ 'Thei^ .'are "dif fi'eul,t'"'t©' lnt©.tp'res'~'^B@cays©'''' ., : 
- they .-,3 re 'based bn.P.^ arid i-lPV responsas combljisdl 



^■fhsn, PV-iv'Py fleets are sigaificsist^ -.the ave'rall sjeafls 
for ea'ch group -are shoiv-rs- - ■ - " - - 1 ~' . . 



> 



ERIC: V 



^ABLE 10 



S.oiirces 'of: Vf^riancoi 



Pro-' s e t: vi ce training 
(3) Total .no. of ho^rs 

For Teache.rvS 

For hid^s 

( 3 ) .■ ; HoM many -days ■ di d ' thei 
.-cover? 
■ •• ror vTea.chars ■ 
■■.For Mdes:-- ■ . ■ ' ■ 



- . Si fces 



43.6% 



12 
II- 




mte% sit- Items aro taken t^om the Sponsor ■Imjxla.iit.ant^^ 

These analyses are foamed only on f iye.^ ij^dije.l^ 'ahd^-'r ; 
because o£ nxisslng data and ^b© omiBsXon)o£' \moMls: ^tth " •\ 
only one sito'o^ Since fchera is only , o:ne.'6bs.er.va t ion v 
\ site., there is^ no within-^sibo variabion ^ ■■'ami .-the' '-s^^^^^^ 
effects cannot' be tescc-do The top io-^^r'ot nm^j'^rs': ior ■ 
each variafol<^ gives the percent of variarice .,,e.Kplained- • 
by Kiodels.amI by sites ^^^^ifehin Modois^ ./'.^Tlie .second r^^ ' 
. . for- c^acb Vi^ri able? 'indicates the- signiflcar^ce' of-^^^^^^ 

ences; boLv^cri modah^^^ Conventional .not^t-ior^ 4^^ useu's.- 



o' < cOOl 

p. .< . OS. 

not significant 



^3^t sitas tV'fe^'a to the dc9):oes of fr^-edoM; for sites? 
^^jsiodcls^ IS not ^hcnvn because is ,alv/ays , ■' \'; ■/ 



\P¥--i«W effects ■ and the, interactions of PV-NPV with .. 
'V'lr^oaels .^and sites. "' K significajnt ,i>V-l^PV effect ' r->ar5s; that 

• di fferences/between,;'t^ t^^o /groups enist wh^en -ail i!K>dels 
a.na sites ara. •cpn5^idBred -tooeVner^ J-f PV is greater than 

■ .^J^PV on a -qimn variable this, indicates that on tha average.- 
: 't\m - BCOTBs f.oT -V^ be higher than the/ scores 
.-s#^r.>'lP¥ teacbers* ■ >rn.,en there'' a'm significant P\M^P,y effects # 

■ the 'tmms- for^/eacli. group are also sho.v^'n. . in] the 7:t&>Iq- ^ n An 
■.Ihteraction, b^^^ FVr-^SPV-. and r^jifees ■ .(or models) ■ that ' 

■ fche relationship be fe^een P¥ -and/ NPV is dif ferent in diffr^renfc 
■'.-sites;:; C6r\ models) .... -The freo,deI,a.nd site . factors .aro -^shown 

only- to display the, .en ti-^^^ ' They "are . o the r.^ise 

■ confoising- because ,th'^Y inclnJa ■bodi -.PV. .and NPV responses c 
-■■ThuSf wha.n. v^^e '-find sig-nJficaTit 'd among models. 

in this' tabla;^ ...it- rrieahs , thf:/fe ' differences -e^o.s t /even ^:d'ien- PV 

• model .dxti^mncBB:.MM.dh }ioM for both^'i^^sperirnental ^ a.nd " 
^control . g.roiJps yis - not :a:|ear*- The^py-^NPV a.nalysas /are, not 

. based op the sam^ ';sitas ^ancl ^ mod^^ l^revious. a.r?alyses ■ * 

• o'n., only ;:Py qlasseso:' \T!k^; PV-i^PV ;a.naly;?^e^' ir^clyde ^ 6 models 

■ and . .12 ■■sites again^ 'iri ba^-anced.; 'design * ' ..This .,nm&er is . 
snrnXler -ttiah tha ■stan<cla.rcl' clasign beeai^se- data, for •nori---. . 

"sitesi!^ (sfe^ -Table 2);* ' ' ., ■ ^ "'y " ' 

, " .Tab.le ..lO., contains. aBalysos ^of -variahco for pj:o-s^i:iv,iat:> 



\ 



trainin-g i terns /ccxken ^ from the sponBot implQ.mm.tation 
report, "In these analyses there "^s only a model factor ■ 
and a. site factor because for each variable^, there is only 
one 'observation, per si-te» Consequently, the' test of signify, 
, Icance for i?5oaei dittexc-nces uses the' ~g an- square for sites 
as;, the deno.mnator» and no ' te.st. of .slcjn.i ficance of the ... 
d.ifferences betweers s.ites is poss.ib.l.c. The analyses are 
based on S f nod els '^ith 2 to ' 4 =s.it.e.s in each. 

. The jneans a^nd" standard deviations for all 3it.e.s for\. ' 
each- of ' thesG^ Vfj^rlables a.ice ■gl verr in hppe.Y\di>t D. ^ The means 
for the models included in. tine ' analyses ©.resented' here are 
sha^m- in Tar^.le il* ' These means' are -shown because they are 
'based only on bx^^b in , the balanced d.es-xg'n ., arid, consequently ' 
are different from the model n^eans. sh own in the apcencxK. 
The site ^neans are not g-ive.n -because .th^</ do nof change, 

The .first .variable presented in -Table , 9 .is the question 
:''Did.;ypu receive pre-sarvice CBu^m>ar)^.,training?''? / 
70% of, the- TciacherB '^ansv/er yes' to this questdon. However^ ' 
bath model and site differences' -are w%.^fdtici^nt. .The 'rnc>del 
Hfeans range fro?^ Mi* fceachors •Ir?* th^ ^9lS^^ mod^l 

' ' ' •• .-: '■ 

The nur&>e,r of si tes can vary h^^ , beeauBct /.:hey are , t.hc^ bas.ic ■ 
unit and not a fxict.o.r to -be axAaiyj^ed. . -.^ - / ^ 

9 ' ■" . ^' . ■ '■, '.^^ ' • • . ' ■ ■ • ••' • 

For csach variable dis.cussc^d^ ^th€ source v;i 11 be indicated' 
by ■an/ a&broviatioJT in : pa.re4ithoses . This' question -Is TQ#4 ? 
Teac^'er :Ques tionnair^ ite;^i nu^e Arr o.xplanation/,of ■oa'::h' ' 
'abb _^iwi a ti on : is \9iven as -a. f.ootnote in accoiTipa^iying tables 
and the aott.ul^te/qxm^t±om^^lr<^B c^n bo -^found in Appendix- 
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reporting that they received pre-service trairiing while 
only 28% of the teachers in the Enable r model . received it > 
it is not surprising th^rt. th^ Enablers are .lowest ori< this 
variable because ther^ is ho^^provision for prjeiservice 
training in this model. In addition to differences between 
models on this question ^ there are also large differences / 
betwee.n the sites within ■ a model . Four models have parti- 
cularly large .'variations between si tes in the* percent of. 
teachers ;whb report that they have received pre-service . 
training: Far West ( Tacoma, 100% yes; Buffalo-r 18% yes ) , 
Arizona ( LaFayette^- 81% yes; Lakewood ^ .. 0% ) , Bank Street 
( Boulder/ 10 0%; Wilmington, 30% ) , Enablers ( Bellow Fall s, 
83%; Colorado Springs and Biilings", 0% ) • There are no 
differences" between PV and NPV teachers on this variable. 

'•'•pA number of the varicibles in this Chapter have' dich'oto-r 
mous responses/ the ^ answers .are either yes/no or check/* 
no, check.. With such varial^les v/e report the proportion/of. 
teachers responding in the text. In the talDles of means, 
however, v/e report mean values. For check/no check 
responses ^ where l=?check and 0= no check, the means are 
directly interpretable as proportions. With yes/no re-., 
sponses, there is a middle step because l=yes and 2=no. 
In the data above, the mean for Oregon is 1.72, which means 
that 72% of the teachers did not" receive pre-service training. 
The percentage v;ho did receive training is obtained by 
subtracting the mean from 2.6. This procedure holds for 
all items v/here yes is coded as 1 and no as 2. . 
■ Because the significance -levels in; this report are pri- 
marily heuristic, we did not perform subsequent analyses 
after finding signi ficant F. ratios. Instead, extremes are ; 
repolfted* ' For dif ference's between sites v;i thin" models , 
then, Wi3 simply report the models with the' widest range . 
among sites on a particular variable , and give the means 
of the 2 sites v;ith the extreme values. /. 
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There are also signi f icant ;dx. rences " among both 
sites and models in response to the qviestion of "How long 
was the pre-service training period?" (TQtfS) ."^"^ Teachers' 
in* the Oregon model report, the highest nuimer of pre-service« 
training days (JC^lO.i ,days)> with Kansas (8,4.) and^'Florida 
(7, 5 ), .following.- At the other extreme, the ' Enabler teachers 
report Bhe 'fewest days ( 1.6 ), and.' Far West is also low (2, 3 ) 
Again, there is a- great deal of variation among sites in . 
the same model. The largest variation is in Arizona . 
( LaFaye tte , 10 days ; ' La]>ewood;. 0 ) / Kansas (Mounds/ 18,5- . 
days; Portageville ; 3,0 ) , arid Ifigh .Scope' (Fort Walton Beach ; 
10 days ; ' Gree ley"; 2,2 ).. /' ^- * ' \ / 

Data on the length' of pre-service training are also 
available from the Sponsor Implementation Report. (See/Table 
11) . The sponsor /reports ; however ; do not correspon(|a^ wi th ' 
the teachers* responses* /The correlation between the two 
accounts of the nunlber of days of pre-service for teachers 



"^■^The analysis of this variable as reported here is -based on 
a response classification s lightly,^altered f rom the one 
obtairied in the ^^actual data. The original data shov;ed a few 
sites with , extreVely high' average^. From the large standard 
deviations '.which accompany those means.; it appears that 
the question may have- been misunderstood by • some teachers . 
In an. attempt to minimize this problem, we receded "all 
/ responses f'of 26 days- or more to 26. J 



4^ 
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is only .02 . -^•^ From the . sponsor ' s accounts , the , di ffe rences 
between, models in the nuinber of 'days of pre-service training ' 
^given are not signi ficant . Oregon has the highest mean 
( 7 > 5 days ) , but^ the mean is lower than that obtained from 
the teacher reports (10.1,) . In the case of .Kansas and Bank 
Street, the differences are more serious because contradictory 
CGncliisions can be driiwn from the two sets -of data. From' 
the teachers' data, the'mean for ■ Kansas is among the highest \ 
( 8,4 days) but "from the sponsor's account it is at the lov;er 
end of the .distribution (4.3) , The high mean reported by 
the teachers .in Mounds may explaii^this discrepancy. There, 
are, also differences; however/ between .the sponsor and 
teacher reports in Bank Street . In this case, the sponsor's 
account of the number of days of pre-service trainings Is 
high in relation to other models ( 7.2 ) , while according to 
the teachers' account it tends to be low (.3 , 3) . Such 
discrepancies suggest that this question has different 



■^^Surprisingly J. the nuriiber of days of pre-service training 
reported by the teachers is more s'trongly related to the , 
sponsors' reports of how much, training was given to the. 
aides (.413). It is not clear what .. this means The 
correlations of teachers' report^ of the length of their 
pre-service. training also- has low correrations /wi kh measures 
of pre-service in days. All correlations are displayed in 
Table 2 , Appendix C. . . ■ ' - ' 

■'••^The Sponor Implementation Reports require a different 
analysis than do the Teacher Questionnaires. Because they 
have only one observation per si te / we can test the' signi- 
ficance of differences between models / bCit not between sites 
Only five models are included in th^se analyses:' REC,: NXU, 
and Pittsburgh were oini tted/because each- has on.].y one site, . 
and ♦ the re arc no data*, f or Arizona/High S'c^pe , or EDC v 
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meanings to different people. As a result/ it is difficult 
to interpret the findings. There are , signi ficant differences 
in the total number of ' hours of pre-^service reported by 
the sponsors with Oregon having the highest number i&Q) and 
Florida the second highest (40^) , 

..■'-!.■ ' " -"• ■ 

PV-NPV comparisons on length of pre-service . (TQ15) " 
reveal an overall effect i n wh i ch j? V te a che rs report more ^ 
.pre-service training ( 7 > 2 day s) than NPV teache rs (3.2) . 
There is also, a moderate site by PV-NPV interaction;^ 
examination of site means sho'.^^s ' that in contras't to the 
overall trend, NPV .teachers 'in Wilmington/ East St, Lovtis 
and Greeley, report mdf*fe pre-'service training tl\an PV ' / 
■tei.achers . 1-*'^ Thus , it -appears that models and sites differ 
in the amount of pre-service'^^trainihg they receiv^e — ^although 
teachers and 'Sponsors dg not agree on what those amounts- 
arer-r-a,nd that PV gets mo.rQ pre-^service tlic^n NPV, ^ 

The teachers were also asked/ **What ' kind' of pre-"Service 
(suiumer) trai-ning havQ you ^:eceivecV and by t^hom?'* (TQIf6)»'^^^ 
Table 8 ?hows that there are significant model di^;fer^onces 
on all types of training as to vvhether or - not the trviining 



:^*No PVr-NPV comparisons are possible fi;om the Spoh^jor 
implementation Roportsy': " ■ ■ "■ ..■ ./V ■ 

-^^0 ax^e limited in intorprcting the findings about types 
and sources of training by the gon^ii'ality of tha labels 
■•u^ed., to -identify some .'.of: these/categories >' ■''Although- i,t. is|"' 
impossible from th^e heading- ^^observations** to kaov< vv'hcthej: ' 



was'given? site differences are significant fox'^ four types 
(lectures.^- individviai meetings^ group discussion, and " . ■ 
role playing.) and are insigtnificant for the. others, •., : 
Examination o£:^ model means revcials that in all- cases , "either 
Oregon or Kansas has the highest proportion • of teacbeTS 
who .report tivey bad the speci fied type of treinino / (See 
•Table 11),' V..^' 

■•There are. also slgni f leant- differences airiong . models . 
on all four categories of response^, to the ••qviestion' of who 
gave the 'training, and' differences , ^among :/.site.s'' v^ithln ■'■/■^ 
npdels. on , tvs.^o; categones\- (sponsor rep reseti tative,', and ..ioqal ; 
> He^^d Start office)^ '....(Table' ■$).■,. ■ • ; Xt .should,"' be noted that", these.- _ . 



IS ■.:(cpnt^. ^."tb: .refers. tQ>: a.- .txainer.-observlng ^tlie^ teadier,.' 

■ ovT-.. :to : the ~ teache r ■;bbservin9:--an exeinplary' ' ■classxoomr ."'tb^. ' • ; ' " ^ . 

. labels .■"for •the- sources; of ^-trainxi^:g '..'are '■more^.-probleinat than,;.-; -• 
the.;: h.<^adihgS ' for. 'ty^es o£ training^ '/Eadv^category;.',h'a0^ ' 
problejois : ■ ■one./ma.jo'rfdi.ffieulty- is the'questi'on';'^ 

' sponsor ■ loc^l^: .irepres<5n.t'ati,ve, is^^ 

■.p.erson;.who., live.s ^iln-otbe- . cbnununity>. -but-- ■Is^-txaibedy;'.; and- i;n 
son^ej ,case,s^^paid.^-by: ;tht;-'^'^spbhs,b ...t^O' b0 ' responsible^' ■fot ; 'model • 
■impleinentat'ion,'-" in that,' site): V. ' ■/'■■I^' -^bo incliided'\ as the^ sponsor' ■' 

■'r-epresentative ;.-qr ;under .the 'loc^l'.Head -Starfc ■office.? ..i^Kev--^ ' 

■ suspectv.that \ different, .teachers'; answered^ the /question'-; in' ., ■; 
diffe:rent /waysv ■ ^ .The;;labei': consultant''*;: is" ^ev^^n ^^ 
because; a; nbtnber' of people.^yca5r,;be;incIud 

.;,t,cint-'^.WhO'^is' sent - to,, ■mqBitor^■x^tpleTm^ 

■.be;,;5givit>g ;a.ny; •traibibg) ', ''the -■fea^:Qrt^ ii^bo'. ■ 

■vorks>\from;:the --;]ir^^ .;or';peopie.'. ^f^pm;.- ^;-' - 

• loc^ar ^bblleces ."•^^Those ■people"::are^^aXl ■le'gitlmatelyl:-knc^il,.a^ >;-, 
cohs;:Ult.ants;/ are'-..ve-i:y'''diffe .p'tliier;^"''.;' 'Ul-th :,' 

' *^otb.er^-drere :;is:.:.no;-;spacc;'-for'^pe^ whom:,.-it :;rQi;ers;'^ ; 

. a. /resul t., clue;'' 8^: /how ■ ■ ■ ■ 

.;categQri^',;-;-. /Thus-^.- .the,' category, iabels :'cont 

. whlcii ■ limit* ■^.tl^o'- xnterp''rc:tations^ which. ;'^can drawn ,:f.ro^u'- .thesp . ■ 

.data* -'^-^ . ^:,-'',.' ■■■■\^:'' V-': .. /\ J' ^'j--- *■■;;• 
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. being' the source of at least one type of ■training; the , ' - 
.variables do not teil- how. snxich or the number of types, of ■ " ■ ' 
. training .given . Again / Oregon .and t^ns^~ have consistent.U'-:; : ' ... 
high: proportions' of teachers who- report receiving sonte : " . — ^ 
_ training ..f.rom. .these people.;.. (Sponsor " Kansas , . 
•2Mi_j^ , ^■ggicfe':::lKansas, ' B7%'^ .;j^g£gglt.Jii) (Table: 1,1) ' ■■. ' ■ 
_Since, these :two - models also have- .the highest nii.wber ' cf' ^ 
teachers ..reporting that-, they xe.ce.ived pre,»service trainitig, ' ■ - 

■ it is possible that th.e , high means for. .types 'or 'sources o.f ' . . .. 
training .are a. reflection of t.hat previous find,ing i-athsr '' 
th^n real _dif a\rence:s;in the types'.oftrainihv^ of fared i To; - 
exaniine- this /possibiNli ty ; &e . a.naiyses 'of A^ariance' Qn ,,;the . / , 
pre~service 'training variables were re-run with on.ly.' -the /' ■ 
responses of teachers './no ' reported .raceiving soms p re-service . 

■training included, .The' results o.e- 'these analyses (See .' T 
TabXe/12 show partial- s of tlVe .hypothesis .e5<;pia.iaing ' 

. ■ , ■•,"'■. ■ ; ■ ■ ■-■■. '[ . ■ ■ ■-■ ^ . " ■'■'.■•:■.'■. ; 

■ "X-^ , ■' '.' - ■' . ■ ■' ' .■• ■ . ' ■■■'■'"!■'■ ■ • ■' '■ ■ ' ' " ' ' 

: ■.^■.■'■0.ar^;^asstimpfcion -;is that ■.teachers who 'did not' ■receive ore--' " 
s.ervice training wi II not: be. able to report - on- the lenothv 
typtv or... source of th;at training.. , These sera. respo,nses'^niav- 
be;- extenain.g -^the range :pf . xesp©n.ses- yin^^ .■ 
beypnd w-hat" it ..vould have been, i i only ■ ti^^^chers who.' had re- ' 

,ce.iyed:^ training: tv^ere -, includecl,-. . This: results .in. skewina'- the: ' ... 

,:iifea;ns yin the : direc tion, . of . those .. nKsdeis and sites " 'in' Which ■ 'sitd re'" ■ 
teach-er.s- reported receiving p,re,'-service fcrainin.9"-to'.fard •those 

■vsrtes; «i^d ■ ittedels:; with: ■:S:eKerv -xeror'responses » ' ' ' / ■ '■ - , ■ ; " , ^ ■:„ .: 

.•^ 'I'he. .;s.ri55pna :and.. £-n3bler moidels ' were : e,xclwaed" f rorn" t 

■4ys^xs ;bec^^^e, i^v;one ;:M:iKona;;si. Khabier ■ '■.■ 

■ S:j;'tes^ tNev?bargh^,:\ Billings ,::/ana: ;G61aradG' Springk^l ^xail^'.teache,^^ . 
....reported, ythat /che . . 
■■:they:eonMtion^|i ysnMyses-,;; these :-R»dels'' would .'«hbslanee-yie: ■ ' .'- 
aes,Ign■ .by; havlngi^empfcy ^:'eellsv.;: -TO'' maintain a balanced -'desiofv- ■ 

, then ^;^:. the., models were omitted ■;f.roK). the ■^vhslysis: ;ie^a^n,e.: 7 ^ ?sodbls" 

■ a,ho,,^^ ■21 .sites. :^ Eu^ir Koi'tvalsb .had om si:te -vitih no ■ore^^'ervice'.''-^^^^^ 
■training ■.,(^res?^b)/ W^e: sjnM'4is"'beeauie'':^ 
: the ..s.ite had- been'-'Qidtted -oreviotjslvv: -■ ■■.■■.■> ■" ■■ ■•".■--:;:;■;.; -^'.. - ; .o" 



■ : Analyses ' of > ¥aria.n.cs :Cl5^iHFngir uoon "^Hecei^vinc? Trai.t:?inC' 
V'.'^^V":' ; 0; Sources- of Variance 
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-Table ,8 with t%»'<3r-- exccsktiohs :. .dsiiy ■'tead5erl.."who. rcoottca 
fcliafc ,fc-he-y - vi3-c€tv^& 9re-sesyicQ--iraiTiii?:9- aye ljic"liiaesS:^-'.;:ahd" 
-as' .a,--rc:sult^ tha. a.salysGS--are based 'on ? -K'todeU- and 'at--: ■ ■ 
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the high means for Kansas and Oregon. For type^ of irairar,g/- 
Kansas has die first or second highest -nK?sn on all "^fotir 
variables for which there ars significant differeiT&'feS' between 
n^dels,. ■^^ relati^l^^ high on two varlai>lss» -bue 

Ll^^MS ^"'^ Ifigh Scope "are also higho^^ Similaciy , ,/:'or 
consultsfit craifiing., • the /one soiitrc© of teaming wifch sigm-^ 
fieanfe Jtodei.diffQ rentes v Kansas has fcho second hiahest 
iftsa?i.C92% of the • fcescl;r4rs .indic^sfce they rcwivea sosire ferainlRg 
'fro?!? a consultant) ai((d HiftL,,lcooe C34f received training from, 
a -consuiJ^ant) > n9t -/0re'gony has the hiohestc^ Tlia 'siost 
Striking fiiiding of the re-ajiaiysis,^ hov/ever^, is the t%d«ccion 
in the Runiber of ,sigifi Sicanc differences arjjoftg sices and 
moGeiSc Kith die responses -of eeachers whc^'recjeived no jsr®- 

: • ■ ■ ■■ <• • , . , . ■ . . ■ ; • . •, ■ - ■ ■)■,•...:,,■. 

service ; craining removed, the - analyses of ■■varis-hc^ reveal- '^'^ - 

„ , " -'^ " * ' . 

. sigiiif leant :diffefenc©s ^. among - models on only ■ fiv©-: typa'S ■ 

and source's of training (as opposed to. eleven significant 

variables with all teachers included) and oniy sifnifieanc 

ttifforencos arnong sices «^ fldreov^ty che level of significance- 

vas raduogd for the significant variables also. Thus, Ur© 

variation between models and ainong _ sit^s vithin R^ydels 



l^Loetiieos Kansas, i;0| Oregon. o9ii V^eik^rt. " .80o 
Disciissicn of video taped ^io^ssonss Keiksx^t^. c94, Kansas? .S3 
Obsesrvationss Kansas, 1.0, Florraa^ c91? Oreaon^ .82 
Role plays Florida, 1„0. .Kansas. o925 Vfeikart, .89?, 
• • Bank -Street, ^ . S3 



■ on the types, -a^d sourcfis'' of |5;fre-s.e,rvic-3 training .is- reducea. 
when only the responses,, of t''eijch»rs who. r.ecei\'sd'',pr©rse^^^^ 
trainina are incJ.udedA? * ■■ ■, 

;■■ ■' •• ■ l^'e _ analyses of .,?V-Ji?V dif fererijs©s . on'. fchev'fey^es: and -. 
•■sources'"' of. preVs'ervi^ce ' .training . variables /■■'■.as- 6r.igi.n'al;l^' '•• 
defi.ns.d- (with all ' fceadiers included) g[ sh^^is^iiflc^ht'',. . ' '' 
overal 1 FV^W . ejiects . f or .^.h.a.I op-t^watishi^s. -but m ; 
few isfce,racfeioh.s with ,sites'o|>i!«del.s 'C^.ee' Tab.tc:. 9 1'-/ fhusf 
Planned Varia'feloR teach^;^ 'reppr.t more -groyp ■diicijss ions # 
discyssion' of ■ m_^©s^^ape# 'i's^js-sons ■ o^.s'rvations'^' and': rui© 

■ blaying than 'do"'nori-'^lan'ned .V^,ri,atipj^,fe©a<Aess/.as^ as 
'tending:- kpre .o£tej5 to rscaive sc^-a .[training-, fro-fe' tlie sssnser 

xesresenfcati^fe .' and „ tJi@-' -%onsuIt'an^ 
..i'ntcrractlons^ '^^xoMn.atian of .sit^ 
•in- s.qfR'e modals moxQ m. -t&&ch.^m than .NP¥-;t©'ac|i©r^ 

receiving ■ lectures;. aiM ,;tral,ning^ from- -the ■\lQc&i'riffcoff ice 



'For th'B] variabi^^s vith-; 




wK'ile in otip£'t modalsy .wiom UPS; than 'pv tpfachera this 
'report? and th&t i ft sowe sites" FV teachers' .receive -mon-? ' 

t?rai«iii^; fro^^ 'lotheri'!' •:.&?•! an do :v:pv .f'&jdWrs'^ while ijj ot*w5r 
.si-^es the finding' is reversed, i FrO'® timse r'/sal.ts, it 

arppears; that Pi a^rned Variation ij:<i'3che:r<i t^nJ to report a 
■ Vfi^er variety_ of tralnirjg- types 1. and soni.^- trailing fr 
■spo-nsors^and ' poriSu3,:taf! fcs ' i.hm ' nt 



n-PJarmGd \?arij!tion tt-acheris 



'M!K£ZJlL»EHl£HZi£!^^ From Table's, i% appe^srs 

/Qf^vafia,fclos^£go;^^ ".^^^5^,. .j!-^'^^'^ wl-hifij ^r^dels . 

,'SS!S^»,lSM..Jl!£.JS£2i^^ Table 9 show,?* overall - 

I ^ V Ji-f ;feireri,ces Oft fsos t vari ab les . S ir^ce the py uji, ' 
Is aaif^ayf::;hi^her tha:rt the J^iPV rj^ean, xhis #incu ao- , can b*? ^ ■ 
taken :tx>-shOT:^,that PiannecrVari^ 3:^a ■ 

•a■;y»^ide^':■-variety:. oC-::p^r&•-s.arvl■c^s• traini-ng; tl^a/t-. do' .noR- Planned 

' ■ ■ . ■ ■ ■ ■. y 

of.iy. ,PV- teacliers aaain, show that, more 'teachers in the 

v- . . , ■ . ■ .' , '.^'-.^i -\ : . v „ ■ ^ • 

Oregon ?n;Odel ■ than^ in ^khje othe'r re;po,rt that thay-, ■■ 

:x^G%iAi&ik■'Vi-.<:i'-sQ^^ixc^ tr'a,ining\ -.The teachGrss in 'U,rcf,goj's and ■ 
&insa&:hav© the sii:6hes0 m0an.s':on l«mgth of; training ^Ctcjachers. 



in Florida also, reported long- training) and.-ori. r,Ype^ ar,fi 
■soarcc?s - of trailing, 7his. B-aqgp%tB that ^he ■ teacher^^ 
these 'models received a viSer yarietv^ of t. ra2.ru.rv^? than 
toacrmrsi/in. 04:-n€r r^KxlfAs.. Of the eight v.:iria&lvK o^} wy-ch 
thero are-' sign! ficant , dif fere?x^s beti^^ovn t.efv , Bank Si:-?ot. 
had Jarge variations^ among -J-'ts sitcrs on'"6 var , Fr^r 
.Vfest on 5,. and Arizona on'4> - 'I 

The rr^ij^jor qu^H f ic^ition to these ' find! ^-scjb thi:?i: >i 

pre-'serviee'' training. If: :m ^::onsid0r the typo^ ^nd sourc^rt^ 
of. training receive^ for only thO'FrQ t^.^cnorq; report "v. 

'pre-'service training, tho di. ffoience^? ba5i:v^^^Qn mo:dai^ and 
p/5ri:ic^Iarly , fehe d.l ir forencet? o^tv^^G:^ w^lthin :nacala: ;>re 

greativ reduced* floreover> on fcho^^e vari^3Li:Ues on which 
.model differences r<^main .significant;, Kan5iias a?^d Gre0on dO; 
no^ always, have the higlu^s't n;eans? Florid^a. and Hi<ih Scope 
alivo. ten:d tb be-.high« .Thus, we can conclude that , the largo ■ 
• differences between models ai^d sices on- assMscts of pr-tii^- 
service training ► are due i:n part to t'hx^ fact that some 
teachers report np- pre--soruice trainiiry:, Thl^^s not to 
■ say that ifc is ??K:>re ' correct to'only include the 'ce^ich-^^r^ir 

who' redaived prB'-sQrvica training in the analysers, but 

. ^ ■ ■ 

' txw^lv that- tha conclusions dr<^vm depend un the perspective 
.. taken i ' - , 



, Tvo Other ■ factOTS must be acknwledged in drawing con- 
clusi ens .about pre-sorvice training.^ First, the low -eor re 1- " 
ation bot'^^oen ■ ^^ponsors * and' teachers^ report^^^ of th^ -inr'^^ 
vari^b^40 can^^ot be ignored. This finding toco^ther .^rith the 
larg<-^ proportion of variance, which, i-s not: explained by <:^ltlriQt 
rmaoiiy or sitos^-^ suggest, that there may be. di iferent urK^:r- 
standings as ?:o >^3^iat these, -questions about in--s^'^rvi 
t.rnu.riing v^^re ;intondcd' to tap* Vague c^ategory _ lMl>eIi> :xr . • 
p^?rt of the roason for -different responses; to the questjo;"^ 
of types' and sources of traini^ng* This^ then, is a |;^econ^^ 
fdccor to 'be kept, ir* ;/xnd in intorpr?;^t in? ' chesc findi; ,> , • 
Because of Ui'^^Sii arri^igu;! :,xes , particudariy in ?,dentifyin^ 
v^ho gave the !;r^^ining^ it v^ouid foe unwi^a to elaborate on 
" thi^ iraplications of the iindinqs .for individual Ya,riabl^:r-> . 
I'hus^ we ca.n conciudo triere are differences hat^Q/^n 
and models in pre-^^sorvice training^' but our knov/Iedgc 
the nature o!:, those dlEferonc^s is , liiui, ted. 

In-^ser vice . training 'may b& more Ir^sport ant tQ is^iploi^ven- 
..tation in Pla^vited Variatiy^n than pre--ser vice traanin^ bec^x^^<:? 
^_^t continueis throughout'' ti^e .ye^r rather ihan occurring for 



'^^This fi:H2.. -ig .has not bccn^ discussed pre vio\Js ly , - bi.'t can 
-found in Ta.:/ ,> S under the eoiumn headod '"Class' v;i thin Si to"* ? 
with '^tVtt- exc.;4pt ions ^ over 60 t of the variance on each variola 
lies, bo twecn teachers in tho same sites/ • * . • 



a ii.mi ted ' t i;r:o during; the Eumrr^r- Tables 13^ 3 'Vf and 
16 vTyrryileA. Tables 8 through 11 for-- pre --nior vice t raifa:! rj-'.^'/ 
Table i3 incluotiifd ^n^^ly'vos of .PV classe?^ only and th^- an-^^:) yi^c-!^^ 
are b/jsed on '3 models ^lnd 27 sites- .in t-he- stano^yr'^il ck/'S;v;n, 
Tab'iO 34 prosents the PV'-NPV coruparisonB for the saire vtiri'^bje^ 
..but with a ^ni^iller s^^irnpic? o:' 12 sites and^ 6 rodci-s". T;^i'>.le i> 
sho^:> <^rH>iyses on .ooni'pfyf'ahle i.tern;:; f ro;n^. the Sponsor I/r^p ] ernen-^ 
t^ation reports. Or^ly ' vari ^iblo^ with yni ficant ,di,f f>.> r(;nco-r* 
?)ire included in any of. -th^so tahlos. Table 16 gives the 
^ncans on the^e variable^i f or ' th'-i^ rrodo Ik inci uded.. in- the ^in^^- 
lyso!^; Ta.ble 2 in Append^: 1^ cont;?(tns the rr;ean:^ ahd iU:£arKburd 
deviation!?: for all site^ arid rno 

^ Since essentially -.al 1 teachers r^^spondcd yes- -to the 
question^ "As you were teachinj^ during the year, war?;- help 
and/or training- available to -Jq-q?'* . (TQ^8) ^ no further analysis 
was dQno> There >^ re dl £f:eir;^nces , hov{0;/er^ in the rcsponsct; 
to'iiie ^.^ccond part of the same que^:-': ion i "If: yes>, hov/ . of ten?'* 

oAle 13.shov;:j^ significant differences ^monq both ^sitos aud 
rr>odclSv / Exa^rdh^ition of Table IG indicates that model t^^Oi^un:^ 
ranga 'ttom a lovv* of I,0'^^or Or-Of|6n (training occurs on the 
• averiioe of once a wc-jkl^"^ to a high o.f '^^.9 lor Kansas • . '■ 
■■ (every oti'iej: rr.onth) . The other !f>o<3oJ.s fialj betvoon traijiiiKi 




...guoGtion is..scalcd-'T;ronel-daily- to 7;-once thi.;. yo'ar, 
so that ^ high', mean roproscnts infrequent trainingl 
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TABLE 



13 



In - Se r co T r aini n g 
Analyses of Va r i an co for PI an nod V a ri a t i ozi Classes 
* ' . . Sources of Variance: 

['ifthi n v;i i;.h j.n 

Varic^blc :^:3jnc _ jMorlel _ nodo ]. 'i^^ J:'- £L!?-^£if'.?J. 

f • 

T0I3 'F'^C'^Vi^-^-nny^of in- 27. S^f^^i 25. 'SOV, ^6.93'^ 110 

Borv). ce tr n i ni ng * 

■ ■ / 

T0^9 Kind of in-service 

tvr cii ning; ^' 
' Dc?.mon B t r" a t .1 on - 1 o r> on k 10. 2^)1' 20. ?3 6B.88 .1 35 

\ • ' . * . ' ' ' • 

Lectures. ]'i,Ofi+ 20.38 ■ rifi.57 v 

i"" G roup d 1 3 c UP. i on .1 u . 5 6 1 . ] B . 0 9 7 ]. . J 5 



Discussion of video- . I4^.,.llt 20,45 65,44 ; 135 



A -A A- A 



Obse rv/:j t ions ^ , ^ 3t 2 2-. ^7 72 .70 | 1 35 



received; 



Rol^ playing ' li;39+ 30.10 58.51 135 

V/ho' yave in-^servico^ _^ / ' - ' 

die fe i'lVj t ty pc /o f. ... 

trainin'-iy given by) : . ' /, 

Sponsor Hopresenta- ' 5 . 84t 1 3.73 75.44 129 

I'ive HS 



Consulrt^nt I2.26t 19.7 3 6 8.01 ^- 129 

Local lis Office 5. 30t' 21.34 76.36 129 



' • Other \ 13.09i- 12.77 . 74.14 ^ 129 

TQ^9 ' No response (kinds 8.421 ^ 24.04 67.54 / 129 

of training not WS .'. 



Note: t Indicates that model effects are not si9nifiGant v;hen sites 

are used as randoiir factors; . See notes for Table 8 for further 
... description of the 'table. ■• , . . 
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■■ TABLE 15 ■ ■ 

In-Service Training 
Analyses of Variance from Sponsors' Reports 



Sources of Variance 



Variable Name 



SIHb In-i&ervice training 

(1} Total no. .of hours 
For Teachers 
For Aic3os 

(3J How many duy • did 
the in-'Service 
t: r-ii/iirrj cover? 

i^ ^ ^r Teauher:3 

* - For 7\; 



( la) r re<j u*vnr», o C 
trai mug : 



ModeJ. 



89 . 8% 



75.0% 
70 . 8% 

* A- 

78. 7% 



Sites 
W,i thin 
• Mode 1 



10 . 21 
9 . 8?, 



2 5 . 0% 

29 :2'i 

21. 3'A 



d f s i. to 



12 
12 



12 
12 

12 



Note: For" a ■.description of thy table, see the note for Table 10, 



'-85- 

















<n 




a) 




o 






r: 








c 






c 






y^ 


4J 








r: 


o 








> 








0) 




a:) 




1 

c 


o 












0 








t: 




0) 




5% 



(?pT.VO I J 



U05v7JJO 



0) 

r-l 

V-t 

> 



ERIC 



in 



o•^ 



CO 

-41: 

a 



<^ O <N O 

vo 00 c m 



<n r"{ o\ CO o tN 

\0 CO 



CO 



in <N 



o 
o 



o"> in 

0"> vo 



1^ <c 



rt ,?^cn r>- m 



f>") po VJp 
04 <**1 CN 



\£) in <^ o 
oh 



f"vi r-i 



vD (N) 
• • • * 

(-^) PI 



a 



O 



03 



m 


00 


o 


(^■^ 




V) 




r- 










• » 


m 


in 


00 












































<n 








(N 




a*) 






00 






in 


("^ 




in 




NX) 


























(N 








^1 


o> 


ro 


o 




o;.-) 


a") 




OA 














oa 













































o 




CN 


O 




(J} 










ri r- o^-) r^'^ 


o 








00 


ON 


o> 






































t>i f\i 












iN 


Oi 


N* 




O 


^1 




r-^ i.n OA 


iN 








oe 




0"\ 


U3 






00 










CN 




















































1 JVJ 


At 


























0 




































































C Q 


' A 






U U) 


c u 








'0 










































c; 0 








Q) 










..A 






(li 4J 


•ri r: 








0, 














U 


n 








(U 








u u 










k o 


U} 






















> u 






















" 0 






0 






0 














> 


a> o 




tl) 




K 


C) 








0 0 


<u 


if] 




(J) li 


O O 


0') 




CD 


•0 


















a) 


0) 














> U) 


•c: 




'J 


ct ' - 


> X1 






o 


*> 








U Q 










M U 


















in o 




M 




0) 11 


c: 












■K 


■ tTr l'> \ 






M l 


(/) H 


0 




0") 


0 














p 


0 <xJ 


- t ^ 


-H 










^» 


C" 


c; 






G 




c 






u 


c: 


c 


0 






a) u 0 


CO 


c 


>* li 






K 


(/I 


0 


0 




>!('■ 


0 


(K c 


' c 




U r- i 




U 


0) 






in 






Si/ A* » 


r4 CO 


0 












'0 
















c 


o 


0 > 


to 


U 






















c: 




p. 












in 0 u 




c: 




''O 0 


,0 




o 


0 




0 


tli 




c: t/j d 0 






o 0« 






u, 


0 








r < 




0 C (J 










a) 


0 


u 
























o a 




o 




■ & 0^ 


10 u. ^.io 







\ 





d 


C) 








.a 


tn 




c 




'.'( 








•rl 


> 










0) 


m 


•o 


0 


CI 














c: 




0 .o 




0 


c 


'V 








c 












in 




c: 








aj 


X 


B 


(1) 


x: 




















t/1 


0 




f:i 













a) 

O 

c 

d 
u 

■1 c 

m 
(/} 



a 
a? 

c: • 

li) 

c 



a 



6 



It 

CO 



ERJC 



0> 

c: 

o 
> 

•0) 

CO 

I 



ITi 

u 

0 

u 
o 
w> 

c 
o 

o 

m 

if) 



.0 



1^ 



o 



U7 



0> 




to 












u 








U) 






■¥ 








>i 




C 


0 ct> 


0 






n> 






jc: 






-^d u \ 












.-it; 
















^4 A ■ 








0 0 


0 




«... 




















0 














u 












.m 




















'i 


.u 




•g 




0 




0 


0 




0 
















i 














c 


»• 



























o 



■'0 



u 



\ 



4J. 



o 



CD 


CO 1 


o 


o 


o 


o 








ri 














on 


... rL 


O 


o 


o 


o 


o| 


O 






CI 


o> 




♦ 


















in[ 


a> 


o 


o 




vo j 























o 

» 



O 

o . 



o o 

■x: u. o, 









^* 

<^ 
















M ... 












.V4 ♦•-^ 
























F'^ O 










*■ 




c 








> 


> c 












a 


<i> 






j!: It 


nji 


1-^ o 






<i> 




o 


0 "v* 




c 








or 










> 












0 








(n 






>li CP 


in 






O 
























0 ^ 














c 




rri 






'Hi 






a? 


: r4 












-r* \ 






c 






(ft ^A-* 






















o 


' U T'* 


E 








V4 




O 










4j. si 














^ ■ 




B 












• 




o 


4^ »1J 



O 



iJ ■ 

. o 
o 

'■(HI 

O 
E 



o 

Oil 



imn th ly ^ .^"l^.^^^. ^!^^,^-^^^^^^ 'There is ^ al so la rge va rX at i on 

among si ten .vi thin the same' model* The sites .in the KvW;sas, 
■RK>del shov^, the laraest variation wifch PortaQcyil le ropoxtuKT 
that in-service traini?>^ , occvirs b^tv.teen onco^a ^^ck^_and o^nce 

^l^QLji:^i£^ ^■'^-^ j ?^£L^ii^^ reporting '<^J^_X2j^'^ ,'i5.3).f. ■ 

and Orailn ro porting approximately ^^^yg^'^ (6,8), 

•• Surprisingly^ there- i^? no di f ference bet wen PV .ind--ti?V 
'on "this .vciriablo? planned V^^riation' teachers, as a group"; ' 
receive' signi ficantly ^^^^"^^ less frequent training tMh\ 
non-" IM. anncd Var 1 a ticn t ea.che rs, ■ (I'ab le - 1 4 ) , The re . is a 
moderately significant PV'-E^l'^yvby site interaction which 
appears to be , due in part ■ to 'one-, site in ( 

which NPV ^peachers report- more frequent training than Fv 
teachers; in the' other suites the.FV v:^ax\ tends . to be ^tQam.t^-:^ 
than or cquai* to the .^^PV ^riean^^ fThe lack of overall. PV-OiPV 
differences X hQwevetf' iTte:a.ns^th,atl tlKvdi f Cerences ci^nona 
models in. fr^?quen5y of. training are not. necess^irily'.'di.ie to'-'/~ 
the experintental fcreatn>ent^ They may instead rcfloc-t di££e-x^-, 
enccs' in the sites other fchan model IntQ/rvention* - 

^- As Kith -fche pro-service train:? n^^ data:^ the. sponsors ^ \v ' 
- '■ 1% ■. 

reports aUout, me ijv-service tr^^ininq. they , cjave, (SI r4b (3);a)) 
do not, correspond,, exactly, vi th th'^ tcache.rs • .accoilntJ5^ of 
ho\>' muJ* they received (See T^ibie IG,) * Althouah '.the data / 
frcrn the sponsors * reports also show'^'"'- "siOTiiticant- di f fxircnces 
a:mong sites :,. :md .models , the' colrroicition, br^tv^^cen' the two '' 



. sources m frequency of in-service training only .32. 
2;H£J.S!£ oniy^ r^oael' for which there? is agrceci8ont. be won 

. sponspr and teachers as to hcv often training was given; 

sources report Uu^t training occurred once a_ ^eek 2.01 r 
this is the hi ate t. mean of all the models,. The discre^pancies 
in the other rriodels suggest that 5:heri> may be diiicxences ' 

in definitions of in-service trainiiie.^ : ' ./■--'■•■■•■, ■• 

The Sponsor 5:mple*T>3ntafcior... _Re:ports als oive inforijuition - 
on the Xotal aumber of hours and clays oi in-service trainin.j 
given by ''fcho sponsors CSl4bU} and (3)), B6Kh "variables 
show si9T:^ific£jnt. differences a?T(ong naodels, •with Bank_ it.r.:r(:?t 
reporting the highest niutdjqr of hours oi training «dven for 
both teachers CX^-3l? hour^? and aides" (262) and, the most 
days ot -^•^^■^-^^9 ■ CLjS£_iSa£liiH 

- is the lo-^^est on. both variables \vi th . hourg ahd-'S diiysfi?,;: ■ 
trainirui for both fctmchers and aides i ' The -'. findinq for the ■ 
Florida r-pdel is unders'tand£ible because che . focus of their 
training is the Parent; .Edyca'tor rather than - the teache'r, ^'"^ ' 
but is not: consistent vith the finding that - the teachers in 
this fjtodel . received a reiarively high 4ifl\ourit of ijre^^service V 

.:tmxnLhq. - ' ' " ■ / . ' ■ 'J . . . ■ . ■ 

. second .aspect of In-service trainino for which there - " 
sre- datsi is the kinds of training received CTQi9):. ' Taljl©- 14 



■22 



There . are no Pi' -KFV. cbmparijsons for the Soonsor Im»leir.ejv 
tatroiv 'Reports.,, ■ /' ■ : ■ - ■ 



sho^^s that chare are significant:, differences Dc>t:h a^^on^ 
i!?odc?ls and, im.onq , site's vithi.n tr^odels on ot the .3 ^typ-x^\?^' 
oi tr^'ini'ng' lis 6 & 'in " the questionnaire C <lefr?c^n^?.v r t: i on .-. 
lessons«, 'l^ctnros i Qtoitp dit^^cassion^srf cn cushion o^: vidvo ' 
taped li^ssons;^. and roiC ^ nlaviag) .and diffenmce^; aJTjong • 
sites bufc not atr^ong rTVLt;^eis on a sixch tv:>o lohservatlott:^") 
...Exaiiiinatlon .^of.^r^a^l:..^anu..Id^- this group of :v.vai4.ab;ies.-..- 

■ C Table 16)- indicates that, mlikQ th<: ^indinq'^■ tor t.ypes. of 
prc-seVvi-ce trcitning, thes^t^m ^tvod^ls' do not ha\^ the hicjhe^^c 
j?^:jans •J:or a).l; fcypi^s or training?. n>Ddols ^vhiclv. are .;:lu-.^h an 

■ one c^pe^ ot'-tr^inlna ^in^iy be tm^ -on anofche^c^^ Th^^s., it ^ " • ^ 
appea^rs tfta^. rr^ade^:'^ •^.■xy ^r^ the, types ...of -training xn.iah ■ ■ _ 
t:eac'he^r^r^^--v-^ve,. ; J^torsovor^: 'tha 'rr^w^a^is- o:f ,fcha ^^.itn^s within 
n;tocelS;.^v;ow; that al^l Cexcapc Orego^i) bav?^ a :a'r^at: • 
(3eaX 'ofjA^arlabilityV: -^^^ on'^at least onez-fcypc^ ox'' ' 
t-irainin^."' : Kansas:^. ;ho 't;he , only::^nod^I wxtrv I-arg^- ' - 

..aisoi^ -the teachers report :th.is 

. kindrof traCm%i .>tound£^^^ had Itl ^ ■ 

^- 251)' ■ I^Yis s-uggesbs ;^ th at ' al fchaugh thero, vis :,va]Eriatioiv '.In - all- . 
^dolSf that Kan^^n^ has th^ mo-^t v^ri^ti^rUi-y between sites 
in^tho typos of 'craiatncj v;hich tho teachers rocoive. ' 

should bo rcn^K>^rcd that- u.-Cf.e -iiui-^rc^ ao uot i ndicat-u:-. 
hcA-; much of: a-^particulir kind nf ti\^ining v;as recoivod. 



yhQ :;l&ck oi overall m^^SI aiilfere-nccs' in types of \ . [''-■'■■'' 
training- is aot surprising because- there 'Is «o reason to r ■;' ''.■.^ 

• • -expect ■ t!tat ■• ■ »1 anned---Vari a ti on s i tc-s ; ait€i--?T50doi'r-tTOtrl:i:- lyc' -— ~;. 

differc^nfc froj« aon-PXanned variation, i^ vthe sams-^&Vo' ^'m!- '.- - 
-did^ h©vo%'^j?r *..,.:«?ftte,cfe. so?rcvc!lff.3rsnces botsire^??^. t ., 
and the control tr^^trnznts. ' It ■i-S;;.su:rtr'£sifta, then,: fehat' '■ ' 

; lessons.. obs©nMtlons/--fe'id^ llie.;; PV-tr. / bv site , ■ 

intersetion is significarit for ^ 
- €siscussidn ,of video taped IsssoRS-i' . Hence,, .it 'appears that ■ th©':;Isck^ 
■ of differences be-t^vsen ■the'-'^KgKi^rissBntal and cos^troi aro«es ' 

, SG«rce -of . tsmningr ■ who ga : 
^service- tralnipf,;;are^^listed/ , 
■ ^^ spon^Qi: xispmmntative,^,cQnB\iltane, local; Hecss Start office,.-. ■ , : 
. a:Ra\'©th©r CTQlSl;':-^'* -^Malyses ■ © ^ ' 

. . - :tnat . i;?>ere are no significant 'VdifferencSls- a?son« mdmis ' in - 

■ ■ _ the nuat^er- of /types :of ;fcra^iun§ given^-W cher . chQ. sponEor ' 
-irGprasent a tlve os^ the local -' Head .Start -of :£ nfeans ' ■/■'■'■■ ^• 

^\;:i-:":- ■ ■ -^i::^'/-'"';\^v--"V^-.-- .'^■v:^-J - v. ^:-': :v; l■.i:-■ 
_ ,- . - 1 : - , y. ..■ . .. .. .^^^.V 

^ ^''Thesc varv'ables have the sa^i^s probieffis of ajr^i juicv os .do •- " 
thQ corrxjsponding br©~servico variables i" ch-cv do not toll how 
•Tfvucii trainina :cach person gavti, Sae footnote' ii, , 0.72 yor a 
mpre detailed discussio.n of those ditrl^cwi tios , -liero -.vore no " 
significant di f i^x-eiicui,, for "indi vidua! n^cfitinas'^; a hioh' 

. ^ropoi'tiost ot toach'^x's-^. in nil report tbii kind of ttainina, ' 




Jll^^£el£,- ©Kcept Florida a.nd;the: &n%.L55r-s,/:; ranf©. arpu^id":;3 ' ' J-' 
Cor the jv^ijftser of ,^aif £eriRi'' kinas';©^ -txaiRing;^^ 



... ..... . .. .... .. 

... — . ...> ■. . ..... ,-„,,..,.;,::. , 

sponsor rc»ffi-se«taay{j » : 'The Jrreans . fivf -vhc M^^id -©f .'^^^ ' ■ 

gis^n by the HS of f£c© ' raft^?^ ' aroiirid ■"g^cj. for ■'ap ymodel:^^ 

di If ©renews, h^w^svar^--' amoito .sii;es wich.in :r^^d@a.S:■':on.•ssQ:fch 

hsvs -lar^ ' v&riatlems./: amo^q: ^^^^^^ ■ Qrg^n^^^^^^^^i 

. tour ijjodols ■ -£££322^ ^SMHSsIi^iil siads o £ ^ 'tx:stnmq aip .-^ v .: ; I:-- 



iv....:;;:.-,- 



.■sugges ts ItHae.; pare ,ar%- Jidcal -di'£FoKeac«<s 
t rai n i fig; gl^eiv^^h i-civ: 

variBtiosi:. :amQ3ig: v; sites -md^lB^^^n ■' ^v^i.M^^ 

;Ol - f i:ve^i, . by . tli^ sponsor -^re^pi^s^ntfiittve ■.s.ugfas-^s 



■ ' 



that- ^ tit®. .sponssrs ds' ftofc:'»i5o>ad«^ .this./ saiie ,iRput;,;t;o-: aii " IS 



. . r/.-.^: ■ ■■ ■ ' ' ..;.v-:-..;': .A" .' ,vr 



. The -nuiTiber bt*kinds,tof training given^b,y the consultant 

•differs significantly among both models and sites within 

models. 'The teachers in the High Scape model report the most 

training from this source, (. 4 . 7 kinds of training given by 

.the consultant), while the teachers in EDC (1.5) and Far West 

, - ^ : — . 

(1.7) . report the leasl^. ^The largest amount of' variation 
among sites is aa the Arizona model ( LaFayette^ J * 5 ; 
.Lincoln , 6.0 ) ^ iTigh Scope ^ (^^Fort Walton Beach, 218; Greeley , 

/ 

5/7)^ ^nd EDC (Patterson; 0.^0; Washin gton / 3,0) also have 
lajj-ge amounts. Tl)e problem with interpreting these findings 



is that"^we do not know to-whom "consultamt " refers. Because 

of/ theT range of people who might be incluvded in this cate'gbry, 
' — . • ' ' * 

.and because of the variation in respcinseb, \\e suspect that 
the term has different L^.eailings to different people. Witi;iout 
knowledge, of those di f ferencea^ we cannot draw conclusions 
abpuj: this variable. This same difficulty holds for the "othe 

-categorv' where there are significanl;: differences between 
models but not between sites. Kansas^has^ the hi-ghest . mean 
(1.7), but the implications of this finding are unclear 

. . — ^ / 

because we have no ' notion bf wTio is included in this . cate'yory . 

Thus, in , looking at the four sources of training, we 
can conclude that there are substai^tial' di f ferences , either 
among models or among sites in the 'number of typ^s of 
training giveir^y each ofi^th^se * categories of people. On 
thpse categories which hav^ sig^iificant mode\ differences, 



different models have high means'; on those cate^gories which 

have significant site differences, High Scope consistently 

• 25 
shows large variation among sites/ •" ^, \ i 

Table 14 shows only three signi'ficant PV-NPV differeijces • 

in the nunib^r of type S/ of training given 'by thesQ' groups , of 

people\ As ^we would expect thewre is a main effect for the 

aponsor. representative with _ PV teachers ( 2,6 ) having^ -a hi.gher 

mean^than NgV tea^^hers ( 1 , 5 ) \ Vfhat is surprising is that 

* 

non-Planned Variation teachers reported any training from 
the sponsor since they are intended to constitute a control 
group which receives no training. 2*^.. There is also a« main ' 
effect for "other," but with the NPV mean (90% of the teachers 



checked "other") being larger than tfie PV mean ( 36 %) . . Since 

we cannot interpret this category i'c is reassuring to note. 

I. - ✓ ■ 

that it J.S checked leas often than';the others.. Finally, 



25a1so, the finding that the ove.rall mea-ns , fou ^all sources of 
training, except "other" are approximately the 'same — between 
2 and 3 types — indicates that the'sponsor may not be the' sole 
source of training. This supports th^ contention 'that teachers 
may be .-reporting non-model' training i-n these questionnaires* 
This conclusion is further supporte4 by , the sponsor implement 
tation report (SlHb(4)), Table 16, which shows thatonly 
Far West reports that the In-s'ervice trainihg^for the model *. 
was given by the ..local staff; >n the other models it was 
'given by the sponsor's staff. , • ' 

^'Despite the overall PV-NPV relationship), in two sites ' . 
(E. St. Louis and Moynds) the NPV meajn ds actually higher 
than the PV mean on this variable. This suggests either that 
the tea'fche^s are confused as to*'who is included in the cJitegory 
'of "&pon??or represeritative" or -that 'the contamination of 
experiraental and control condition? ih^the^c sites is ^ 
tremendous. 



there is a 'site bV PV-NPV ihteraction . for, the" consultant : 
in 7 sites ^ PV' teachers repo'rt mor^ kinds of training from 
the consultant than do NPV, and in 5 sites; the • r'elationship " * 
-is reversed. , . " ' ^ * v * 

^ ' There is one additional variable, which can be developod 
from the question of what kinds of train j/ig were received 
and who gave them. This variable, no response, indica'tes 
the number of types of training not^ checked as being given 

by anyone, and if the order of the means aro reversed^^it 

\ ■ . ' 

gives a ranking of sites and models asyto where the most 

types of 'training were checked, A high mean in this variable, 

then, means that several .type^s of training' wpre not *given. 

Th.e analysis reveals no. significant' model differences for 

this * va^ri able (Tab 1^ 1 3) ; indicating that Ijio model 'gives a 

substantially w-idep: variety of training than, another . There 

also are no PV-NPV differences (Table 14") "Phere are; 

howe>>er, significant differences amonq sites within m^odels. 

Three^models have ' particularly large variations amonq sates: 

. ^ ^ . ^ . ^ ^ ^ y ^ . 

Arizona (Lincoln, O.SJtypes of training noT checked; LaFayette 
\ \ ■ ' ' ^ 

3.2), Kansas (Portageville , 0..8; bounds, 3.5); and Gordon 

, 2 ^ ^ _ . ^ . 

l^ltouston, 0 .7; Johesboro^ 3.7 ) . ' 

Summary of In-rservipe training : Since no one Or two models, 
were consistently high on all in-service training variables, 

the model 'to m6del differences * (Tables -13 and 15) sojggest 

' ■ — ~ ■ — - - ' - _ V 

that teachers in differjent models receive .diffcirent training ^ 
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in terms of frequency ^ types ^ and sources, Oregon repo^^ts * 
•the most frequent in-service training and tends more often 
than dther models to havfe demonstration lessons and group 
discussions. Bank -Streetx^epprts the most hours and days , 
of training and seems to\4void lectures. Kansas gives more * 
demonstrations than other models, but fewer group .discusssions 
'-Fai!', Wes t and Arizona give more lectures and discussipns ,of 
video tapes , and Far V/est also does more 'role playing. There 
are no - signi f icant model effects on the number of types .6f 
training * gi^ven by the sponsor or by the 'local .US office / nor 
af^ there mo.del differences in* the variety of trainyig* 
received (number ®f types of training not checked) • The 
finding gf training being gi^^^ by people othe r than the 
sponsor indicates that the teacheirs are" receiving non-mode^l V 

^ . ^ — ^ f 

trailing. » - ' , . , » . \ * 

The significant site effects emphasize the importance 

- , , _ ^ i ^ ^ . 

of reeognizing/ that ^teachers with in 4:he same model ^report 
yery di f ferer/t --fe^ining experiences.,. From this d'ata, however, 
it is not possible to de termine 'whethe r the di f ferences 
result from differential sponsor input to the s^tes or from 
non-spQHsor input. ' ^ . . \ 

\ ^ The failuj;?e to' fi nd significant FV-NPV differences 
(main effects or interactions by models*) ofr 'ove r h-'alf the 
in-servicc variables is important because without di'ffercncos 
between experii^iental and , contror groups , ' we cannpt responsibly 
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draw inferences about the effects of models. These findings 

. c . ... 4 ' , . ^ 

' ■ r / . 

may be explained in tv)o ways. The lack of; differences may 
» . ■ • 

be the resuftt. of contamination:- non-Planned Variation / 

- - - - - - • * ■ ■ V ■ ' . ' 

* ^ « 

teachers as well as Planned Variation teachers are being .> • 

* ' /, ^ . • 

exposed to model trainin^^^^^^The fin^din,g' that NPV teachers 
report training by the sponsor supports thi^s view. 'Or, ^he 
effects of |nodel and site differences for^EV only may 'result( 
from * differences in che site.s rather than from differences ^ 
in model treatment. In this cas^ , 'we would expect either 
that PV-NPV differences would not occur, or that if thev did 
'occur, they would be unique to the site. If this explanation' 



dio 



is. correct, it may* also be true'that model-- related treatments 




b not vary fron site to site* ^ . 

In usin^^this iriformation on in-service training, it 

* ' ■ ji 

iriust also" be i>ememberpd that iJL has the same problems that 

we-re present for the pre-rservice variables.' Ill* addition to 

t « m 

the *non-model input mentioned above, we must acknowledge 
the limitations which result from vaguely labelled categorie3 
and from .lack of information on hoW^ much of each kind of * 
trainin'g was given. Because of the problem of ambiguous 
categories., it is possible that some of the variables, 
particularly the sources of training, have 'different 
meanings for (different teachers. This possibility is 
supported by the finding of only moderate correlations betweein 
sponsor and teacher reports, and by the large proportion of 



variation which lies among 'teachers in the s'ame site and 

|is not ekplalned by site 'or model effects-. This p*roblem 

* ' / . * .' ' 

indicates that we must be cautious in drawing specific 

inferences fjsom this data. The lack of information on how 

■ . ' 

much, of ^ each type of training was given* is limiting because 
it 'plfecludes tbnclixsions 'about the* relative importance of 
differences "in' the , l^inds . of varialDles. 

, The mo^fr striking ^findings from this group of in-service 
Variables then, are the large number ^f ^significant differ- 
enc^s between .sites and models (although* they are . fewer than 
for the pre--service variables) , and the relative absence of 
TV-NPV differences* , • " 

Continuing Support and Feedback : .... 

^jfihe frequ(^ncy and types of training given are not the 
,only aspects of training which may vary within Planned 
Variaticjji. One additional aspect ""may be loosely de'Sined as'^^ 

• ' • . . . 'r 

continuing support and feedback. In identifying this area, 
we are^ proppsing that formal 1 training sessions may not be 
a sufficient basis for implementation of a model program./ 
Teachers may ^Iso need help with speipific problems and 
quesi:ions, *support to bolster their confidence and indivi- 
dual feedback on their p^r foriftance . This aspect of training, 
then, will considered in this section.' 

In addition to daLa from the teachers' report^-, there 
are four items on support cind feedback in the sponsor and 
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the consultant "reports . ^'^ ' The format for presenting the 
data is similar to^that of the ^revioug sections: Table 17 
presents the analyses of variance for the Plann^.d Variation . 
classes^ Table 18 shov/s PV-NPV comparisons, and 'Table 19 
shows the means on these variables for the models included , 
in the analyses. Tabl^ 20 presents the cori:erations between 
the support variables^. ^ ^ 

The first question presented in Table 17 is, "To whom 
do you go (most Qften) for help and infdrmation in implement- 
'ing tTie program?" <TQ#12} . Of the-^four response categ'dries 
^^nalysed, three/show significant differences among model's 
and among sites within models. 29 on the "Sponsor 



^^One variable from the. TeachervCuestionnaifre , satisfaction . . 
with training, which is included this section is more 
appropriately described as a reaction to the training rather 
thcin as an- indicator of what 'support .was -given. When v^e 
examin-^ *our theory of implementation these two dimensions 
will be separated. For p\ rposes of data presentation, however, 
it is more instructive to* con±)ine - them. 

'v^^The original questibir has a ififth category, "no one available.' 
.Sirrce only 2 r-espdnses of a total of 448 were„entered under 
thi^ he.adf ng , it was dropped from further analyses. The "other" 
category was analyzed but showed no significant differences » 

^^Again we are faced with the problem of vague categories. In - 
mdst mod.els, primary responsibili'ty for model implementation is 
givepo^ to a loca,l person wh^o* is trained by the sponsor and who. 
works' full time in the si^e. is not clear, however, where 

^uch a person was considered ^in this question. It is possible 
that st>me teachers,; included, her in th<^ sponsor representative) 
category while others may' have included her as an assistant 
director or even as "other. 'l ' ■ . * \ 



1 



ERIC 



TABLE 17 



* Continuing Support and Feedback . * 
^lyses of Variance for Planned' Variation Classes 



Sources of Variance: 



Variable Name 



Model 



Site 
Within 
Mode 1 



Model 
Wi th i n 
: Site 



df Cla&s 



TQ#12 To whom^dp you go most " 
often for help. in imple- 
'menting the program? 

Sponsor Representative 



Head Sta;:t Director 
Another teacher 



11.84%+ 

8.97t 
NS 
10- 70 t 



"9,58 
NS 
22.15 



63,79% 

81.46 

67.15 



135. 
135 
135 



TQ#13 ' Did trainprs stay long 
enough to be really 
helpful? 

TQ#16 ' How satisfied were you 
- ' .with training offered? 



12.23t 



16 .20 



17.21 



11.06 



70.56 



72.75 



126 



127 



SlS^m, Sponsor feedback to 
teachers 



42.1 

i 



57.9 



(df Site) 
•12 



'Note: SI:- Sponsor Implementation Report ' ^' ^ 

+ indicates that mpdQl e.f f ects 'are not significant when sites 

are used as random factors. See notes for Table 8 for further 
de'scriptiipn of the table. 
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RepresenrtatpLve " variable. Bank Street and Florida ' ave the *: 



lowest mean 



s - with :no teachers going' to the. sponsoi?^ most often 



for help.'! i' ar Wes t Oregon , and Kansas have the highest 
means (- 32% , 33% / 38% respectively), but even in these models, 

only one-third of the teachers report that the sponsor 

■ ' . \ , * ' ' f , 

representative is »th^ most ,frequekt source of assistance. 

\ - t, . 

There is also Uarge variation ara6ng^. sites within- tnodel's on 



this, variable Ali:hough ' Arizon,a 
variation airtong sites (Arisona: 



jahd Oregon have the^argest 
Lakewood, 75% of the • 
t'eaG(ners go tiO the sponsor rep, Lincoln and LaF ayette ,\p % 
go to the" sponsor; Oreg'o n : Tupelo, 75%, Ej, Las. Ve-gas / 0^) -, 
five additiqnal models (or a total of. 7 of the 9 models ■ 
in th^ analysis) have at least one s'^te-in which nb teachers 
'check the .sponsor representative on this ' variab'le,^ 'These 
findings are surprising. The low mode iMneans are Oontrary 
to the expectation tK^t the -sponsor representative wx)pld' 
serve as the primary ail^thority- for implementati^., In part, 
howeveir, the model meany are misleading because oV the large 
variation among sites. \^ This latter finding is perhaps 
radre interesting because \it sugges.ts either that sponsors are 
varying theVr- input from $ite to site, or thatS^eir input 
IS .the same but the teachdrs. aro perceiving ^.t dif fe-rentlv . 



The other source of help with significant efKofCts for 
both sites and mode'Ls is "Another Teacher," Cohsidejing tlie 
differences among mode"-\s, Oregdr) "'^nd^ EDC Have the lowest. ' 
m^ans v/ith 0%_ of the teacheirs goin(^ t6 another teacher most 



often for help, v/hile/ Florida has -the ,^-ii:^hest with 41% . Kansas 



Florida have the hj/ghes^ variability apong" sites ( Kansa s : 
in^ Pjorta ge vi lie , 75% of the' teachers go^jjj^st often to anot-her 
teacher for\help, in Mounds and Oij-aibi , 0% do; -Florida." in 
JonesborO; 6 7% go* to another teacher, in H ous'^n 0% do) • « 
There are no 'significant model or *site ef fect§ for 

choosing the Head Start diiiectcr 'but the variable was 

■ - • / • 

included because of fV-NPV, d^f^Eerences • The overall mdan for 
this variable indicate^ that 20*|: of the planned Variation 
teachers go to the • Head \»tart director most often for help. ' ' 
This is approximately the s.ame as the proportion which go to* 
tie £ onsor representative most (18%) and slightly more than 
the proportion who go 'to another teacher ^ (14%) or- to ot^^r (15 
. The.PVrNPV comparisons 'on these variables show overal]* 

c , . * * ^ - 

PV-NPV effects for the sponsor representative and the .Head 
ti • . ■ 

*- * ' 

Start director, and interactions with sites or models for 
the sponsor representative', the Head Start director and 
anothci teacher (Table^ 18) • / For^t>he sponsor representative, 
the overall effect indicdteB that PV teachers tend 'more 
"of^':.3i to- go . to tfiq sponsor mb^t frequently than NPV teachers, 
which is /to be expected, alt'hough* the PV mean is not Igirge, 



On the other haffd, *^e PV-NPV by ^ mode L interaction- on ttiis* 
variable indicates that, this felatic^nship^oes not hold 
for all models.. The PV-NPV analysis for another teacher 
shows no . overall effect but a strong interact>ion with models. 
The Head Start drrector variabj.e shows a Strong overall *'^PV- 
NRV- effect ( PV=15% of the teachers, go to . the Head* Start 
Director most often, NPV=51%) , which migh^ be'explained in 
terms of mon-Plaiined Variation teacl;iers having fewer , ' 
alternative so.urces of help than Planned Variation teachers, 

^ V 

At the same^ime, there also is a PV=NPV by site interaction 
which complicates -'iniierpretations •• • ^ 

It* i.s apparent, then, that taere is a. great deal of 
variability in- whom teachers go. to most' freguentiy for 
help in'i mplementing the program. It also appears that 
overall, teachers are as apt to go to the Head Start* director 
as to the spfi^p^^iklrepresentative . PV-lSIPV comparisons s-how 
that py teachers go moire often than NPV teachers to the 
,sponsop ^presentative apd vice-versa for the Head Start 
director althd)ugh the^re are interactions with- suites or 
models oh both variables. ^ ' """^^^^^^ 

Another question which is relevant^ to the kind of 
support which teacherS-^are "^g^^tlng is ''Did those who trained 
•you sta:^ long enough to be really helpful to you?" (TQ#13) • 
The analysis .of variance on PV classe^ shows significanc 
differences ampnc both models and sites within models 
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f (Table •I?). Examination bf the model means (Table 19) 
:in<3icates. that overall, teachers* ^tend to respond ye^s , the 
trainers did stay Tong enough. In 'the Florida and High Scope d 
j models*,- .however, 4-2 % and 46 ^ - of the te^'achers , respectively, . 
report, that their traineis did not stay lon.g enough. Thi? 
suggests that, according to this measure at least, support 
^is inadequate in thoso models. The variation within models 
y/is highest in - Kansas ( Port age vi lie and Mounds, 0% of the 
. teachers say trainers did not stay long enough; Oraibi,#60% 

\ . . ^ A ~ ^—x- 

say no) and High Scope ( Fort Walton Beach, 0% na; Seattle, '67% 
no) . This indicates that trainers are judged differently in 

• di fferent -sites . Whether this is due to differential input 

I i 
^ or to different receptions is not clear. i^or is it clear 

that the- trainer ryeferred to is always the sponsor or his 

^represenl^^ative . /Moreover, there -is almos^l^^io overa^ PV~NPV 

difference on. this variable (Table 18). Again, this raises 

^problems for interpreting model^ifferences (i.e.^doHo PV~ 

NPV differences stem- from coiitamination , or are model 

differences- the result of factorfe unrelated to the model?) . 

The moderate interaction with the site factor suggests ^^^^^ ' 

the PV~NPV differences which' do exi^ 'are the result^of 

local site conditions rather than ofyPl^anned Variation 

treatments. 

i'- •■ • ■ ■ ■ • 



Sourdes other than the Teacher Questiofinaire provide 
data on suppbrt and feedback. B'oth 'the sponsors and the O'CD 
consultants^ completed site assessments containing -two 
relevant ite;ms : availability of sponsor and -sponsor feedback,' 
to the teacher (items 3-1 and m i\^the sponsor reports and 
items 13 and 14 on the consultant forms) . These are the \ 
most direc|: measures of the > dimension we would like to tap, 
and they provide perspecftives .different from thos.e of the - 
teachers. Analyses o^ variance, however, show significant 
^differences among models for only the sponsor's rating of 
the amount of feedback they give to thq^ teachers. Table 19 v 
shows that the high mean on thip variable is for; Bank Street 
( ^.0 , high frequency ). L . 

An additional support I variable , taken from the quest^n, 
"During the pas^t^ year have you personally requested help or 
training?^' (TQ#]L1 ) ,^ shows no significant differences among 
models, sites, or PV-NPV. Essentially alL teachers, Both 
PV and^ NPV responded yes to this question; the overall 



*^Dif ferences l/etween sites cannot* be tested because only one 
observation pet site was made, ^and^ sites, cori'SequentLy , are 
the basic unitVof analysis. Using the site as the basi<: unit 
also means that^the number of degrees of freedoni in these 
analyses is sm^ll. In such a -situation it is diff,icult to 
determiner >iyhG the r the lack of significance on tK^ other 
variables is due to a real-^lack of differences between, models 
or to the low pow^r of the tests- (power (1~B) is* the probability 
of finding out that the null hypothesis is'^wrong). Mo PV- j ^ 
NPV comparisons are availcible for this data. 



proportion for* PV teachers is 90%. The variable is not 

included in the analysis of variance tables, but the model 
* r 

'31 - ' 

means are shown in Table 19. 

We have .presente^d a number'of different variabl^^ which 

wpre intended tc5 measure continuing support and feedback as " 

a dimension of sponsor injj^ut. Given this intention, we would 

expect the variables to be highly ' correlated. In fact, however, 

tiiGy are only modcratolv correlated. A discussion of theso 

correlations can be foupd in Appertdix C, Table 3. 

Finally,, there is a related but distinct variable which 

can 'be included in thi^ section: "In general, how satisfied 

are you with the training offered you during the year?" 

{TQ#16; l=very satisfied; 5=v^fy^ dissatis fifed) . This ^ 

question is probaWlv best interpreted as a ? general reaction 

to training rather than as a speci^^fic* indicator of. support 

, ^' • . " ^ 

and feedback, but is interes'ting in the present context. 



Table 17 shows significai^t cAf fere rices ai^ong models but 
not among * sites within models on this variable. The* 
teaphers in E nablers (X=l. 6 ) , Kansas (1-. 7 ) ,and O regon (1.8 ) 
have the highest mean levels of ^satisfaction The 



31 ■ ' 

Although most teachers ' reported requesting help'^, we i 

hypothesized that there might be di fferences among models or f 

sites in the number who actually received help.' To test / 

thi3>« expectation, we attempted to analyze the number of ' 

;:^tiines teachet^s received help as a proportion of the nujT±)er f 

of times they- requested help (TQ#11 bf a) . The analyses, 

however, faile(^ because a nur.±)er of teachers responded ' 

inappropr<atGly": they recorded that they received help 

more times than they requested it. 

/ . 
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othev^ models arq. als^ fairly high; Florida which has the 
lowest mean (2 >_8) is in the range of "somewhat satisfi^. " 
^ Comparison of PV and NPV means, however^ shows that there 

are no 'differences in the overall^ means. The finding of ?no 
overall PV-NPV effect dndicates th-at F^lanned Variation 
teachers as a group are not more satisfied with their 
^ training than non-Planned Variation teachers (Table 18) . The 
significant interactions indicate that in some modeis and 
sites, non-Planr)ed Variation' teachers are actually more 
saftisiied with their training than are Planned Variation 
teachers. 



Summary of Continuing Support and Feedback : In this section 
an attempt was made to tap an aspect of training other than 
straightforward accounts of the types and extensi veness • 
J One Important finding which irfesulted is "that there^r are 

differences amoi?g models ' and even larger difference s among 
sites within -models" in the numb er of tfeachers who go to • 
the sponsor representative most often for help in implementin g 
a proqr^am . .Because the sponsor representative would be 
expected to be the authority on the model, it is surprising 
to find that in some sites no teachers go to the sponsor rep 
most often ahd that, overall, teachers, are as apt to go to 



the Hea.d Start dire ctor most frequently as to thc^ sponso r rep< 



Mort^^A^er, the PV-NPV by model interaction on this -variable 
indicates that in some models PV teachers do not go to the 



1 
I 



\ 
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sponsor rep more often tha^ NPV teachers. These findings 

■may be. partially explained^ by /vague categoi^/ ' f ab^s , and 

by the 'fact that the Head Start dj.rector is always present 

at the site. ^ ' 

A no£her interesting finding is that the teachers are 
% *. ■ . ?. 

generally satiS'fied with thei r training. There are significant 

♦differences among, models on this variable, but even in the 

model w.i th the lowest mean, training is ra''ted as somewhat ' 

satisfactory'. There are, however, no overall PV-NPV 

differences in satisfaction. 

The 'Florida model is extreme on ^nore of the' variables 

analyzed than the other models. We find that it isychairac- 

terized by teachers not going to the -sponsor representative 

for help most ofteh, but instead, going to' another teacher.,: 

(although there *is high vairiability >7ithin the model Wiethe 

latter variable) . Moreover, Florida is low on satisfaction 

'with training' and high in judging that trainers do not stay 

long enough to be teally helpful. We might Cvpnclude, then, 

that the teachers in the Florida model do not receive ^ 

strong support from the sponsor. This is consistent with 

the model's emphasis on the role of parent educators rather 

than teachers. Other models do not present as clear a 

piciure, but '.are ' extreme on seve;ral measures ? in Bank 

Street; the sponsor reports that she gives ^- great dead of 

feedback tso the teachers 'but none of the teacher's go to ' 



the -sponsor representative 'for help most . frequently . The' 

teachers in Kansas and Oregon report that 'tbey . are quite 

1^ * 

. \ 

satisfied with t:heir training.' (although in Clregon,^ the 
satisfaction of PV teachers is not greater than that of NPV 
teachers) and more 'of them than in other itjodels gq to the 
sponsor for help most frequently (Oregon also has high 

variabilx'ty among sites on this variable). i^oreover, Oregon 

■■^■^ ^ ^ • " ■ . . ■ a 

has a low mean on going to another teacjier most frequently, 

and Kansas has large differences among sites in the number 

of teachers who go to 6ther teachers for help and in whether 

the. teachers felt that the trainers stayed long enough to 

be really helpful, ' ' T ' ' 



In interpreting -^hese findings , two additional factors 

• V 

must bfe considered, F:^rst, low correlations b,etv;een the 
support measures sugg^es\t that t^e dafa may be unreliable, 



and second, the lack of .overall Py-NP;^7--di f ferences indicates 

tha;t Planned Variation teachers do not re^r^i've^'"^ support 

than non-Planned Variation teachers in all cases. * 

» # 

Sponsor Input Variables as Det'elrminants of Implementation .; 

^In attempting .to explain the variation in levels pf 

implementation, we propose that the variables just pre§.ented, 

the length, types and sources of training as well as support 

factors, will be important. Mor^ specifically, we' expect 

that more training, both in-service and pre^service, ^ and , 



mo 



ore support will^.be related to more successful implement- 
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While we cannot make similar simple statements' 



about the relation between kinds and sou:;ces of training 
and levels' of implementation, we do not fee-1 they are un- 
important: *we expect that suqcess.ful implementation is 
related to an interaction of the "right" type of training ^ 
at the "ri^ht" time; with determination affright depending, 
.no doubt, on the people involved and their stage of train- 

'in-g. We also expept that more training from the sponsor 
shou.'.d lead to better implementation because the sponsor 
knows tAe model best. Butf merely giving more types of 
^training, as is reported here, is not necessarily related to 

Tiffuplementation . Finally, we assume that training giv^n by 
the local Head Start office or by a consultant may influence 

» if 

implementation,, but without further information about the 
-nature of the training, it is not clear whether it would be 
a help or a hindrance. . We predict, then, that if these 
\/ariables in^part determine the extent of implementation, 
. then . variations in them may help to^ explain variation in 
levels of imp'lementation . 

But while these varialjles may'be partial determinants 
of implementation and Vhile there are more data available 
for sponsor* input th^an for th? other areas to be discussed, 
we would argue bKat additional data on sponsor input is 



32Terms such as successful or better implementation are used 
synonymously with a higher level of implementation; all are 
defied in terms* of the objective of fully implementing the 
mode 



necessary for an adequate study of Mmplemeijitation . One 
reason is the previously discussed problem pf vagueness and 
unreliability in the present measures. Second, there are 
additional, dimensions ,. of sponsor input which may affect im- 
pleme^ntation but which have not been studied. 'Since we pre- 
diet that these dimensions vary among sites, we e^opect that 
they may help explain variations in levels of implementa^tion . 
The need for studying ;thes,e additional aspects emerges from 
the present findings. First, the da±a suggest that input' 
varies, but beyond the -«:l^e that iponsors do not give all the 
training > it does not indicate why. We do not know whether 
the input made by people other than the sponsor's staff, or 
non-sponsor input,, accounts for all the variation (with 
sponsor input remaining the same to all sites) , or whether 
part of the variation is -the result of sponsors .changing 
their training from site to site. Second, we do not have 
full knowledge about who \s giving the training. We do not 
know how the local sponsor-\representa^ve fits in; nor do 
we know how much of the total training in a site is given by 
non-sponsor people, or what kind of training they are giving. 

These questions suggest two broad areas of study. One 
area is the sponsor's organization, or staffing pattern. Thi 
concept includes the rtespo^sibilities and expertise of the 
staff involved in transferring a model from the sponsor to 
the teachers ,in a Head Start site r and the relationships be- 
tween them. Anecdotal evidence frpm consultant reports and 
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from conversations with some of the sponsors underlines 
' the importance of this concept to implementation, 

■ One important position in the path between the sponsor 
and the teachers is , the member (s) of the sponsor's central 
staff who visits the feit%s. Beyond the general conditior\s 

that the sponsor's staff" or representativ^es do* visit , other 

V 

factors vary, sometimes within models as well as between 

them. ' One such factor is the size of the sponsor ' s staff : 

the number of people responsible for 'implementing a model . 

at the sites. Another factor which varies is arrangiements 

with the sites: the frequency of sponsor visits^ and the decis 

as to whether a site is visited always by the 3ame person or 

* ■ • • • " ■' ' ' 

by a rotating staff. A third, less tangible, but perhaps 

more important, factor is the role of the sponsd'r's repreS<entrt 

atives . ^Variations in the ^^nsor's role were 'discussed in 

Chapter 1; anecdotal ^e^idence suggest's that in s^ome sites, .// 

• \ 

the sponsor's rej^resentative niay be concerned- only with' 
working with the teachers tc( implement the model in , the clas^s- 
"room, while 'in others he may become involved in- a much wider 
range of problems. We might predict that^in situations where 
the sponsor ' s representative was more i'nvolved, the teachers 
' be more. responsive to the demands of the model and might Vork 
hardsr to implement it.- If this, or some o£h[er relationship 
with implementation exists,, then variation in this fattor * 
might help explain variation in levels .of im^lement^iXn . 



33The issue of 'Variation in the sponsor's role, is important 
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Another key position in the path betv;een. the sponsor 
and the teachers is that of the local person who is rppcns- 
ible for helping ^:he teachers with the model on ^ daily . • 
basis. A. person in this capacity- is present -in mosto models^ 
but* the specific arrangements for the position var^., ^ 
some models^ the local sponsor representative is given; 
special tjfai^ing, while in others she receives the sanle 
training as the teachers. in some cases, the locail, rep- 

resentative is the primary liaison , with the model, with 
the teaching staff having little .direct- contact with the 
sponsor's staff, in others the • sponsor ' s staff provides 
the bulk-of^the training directly 'to the teachers and the 
\^local representative provides support "between spcinsor visits." 
^ The importance placed on this position also'Varies from site, to 
site. ' In some sites, working with the. model is the local 
representative's only i;esponsibility . In .others, the model 
is only one duty among many. We would expect variations such 
as thjese to have an. impact on impiem.entation . If this link, 
between the sponsor and the site is weaki^ implementation will 
suffer.. One OCD^ consultant asserts, fot example/ that the' 
local representative 

"is weaK and needs strong support from the mpdeller.... 
Teachers are hot given consistent help in deepening 
• their understanding of the m.odel . . . .The modeller" does 
not have field support . " " . , ' 

r . 

not oniy to explain' variations in levels of implementation, 
but also to define the treatment: if two models vary on 
dim-ensions which might affect' child outcome measures, then 
those variations should be documented. 
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Or, in^an even more extreme^ situation, an QCD consultant 

rppoirts hds conversations v^ith vteachers aftex the local 

representative has returned from a week Is. training session 

at. the sponsor's home base: " . • . ' 

"When I asked what sort of fqllov;-up the. teachers 
. had, two could remember none while one teacher 
could vaguely remember 15 minutes being devoted 
to* it at one meeting but couldn ' t remember ' whiat 
wa's covered . " ' . 

The consultant concludes that "•...training carry-over does 

not seem to be being;^ accomplished . " • \ 

'■■ ' ' • ■ . •' ■ • ' * , ' '\ 

It 'would seem that in models where the ^ sponsor ' ^ staff 

attempts to train the teachers directly, the loss from a 
weak local representative would be on-going support 4^d the 
answers to practical, day-to-day questions. We mighj: pre- 
dict that in thA.s situation implementation would be jjiffi-^ 



cultv but not ^impossible . On tl^e other hand, in th^ models 
where the loc/al representative receives the major portion of 
the sponsor's tr-^ining, and has primary responsibility for 
training^lier staff, an ineffective person will be. disastrous 
for implementation. ^ x 

The importance of the concept of the sponsor's staffing 
pattern can be summarized in ^ statement b^ one of the OCD 
consultants about the problems which a breakdown in this com- 



•''^Anothex person who may not be directly involved in the 
training path between the sponsor and the teachers, but. who 
is probabl^y .crucial to implementation, is the Head Start 
Director. The role of the Director will be examined in 
■the sectibn of this chapter pn Context Variables. 
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ponent creates: ^ ^ * 

"The major problem [this site] is facing and yet 
which is the key to successful implementation of this 
type of model is lack of communication from top 
to bottota. " 

,And in another report,' the same consultant commenting on t^he 
same site.tstate^ that."*The communication patli seems to stop . 
before getting to the teacher." Variations in this concept, 
then >^ may explain variation, in levels of implementation. 

A second area for which the data*^ are inadequate is . 
inputr 'from' people other than the sponsor ' s * staff . Since 
our data ■ indicate that training from people other than the 
sponsor ocgurs, this area should be examined. While it i^ , 
possible that these non-sponsor inputs augment the sponsor's 
training in the model, it is also possible that they are 
totally unri^lated, and perhaps even Confuse^ the sponsor's 
training. It is important, then, to determine who the non- 

'V 

to the teachers, and what the content is. In relation to 
the last question, we are further inter'ested in whether'^thfe 
input is formal training or assistance v;ith specific prob- 
ietns; whether training is in one -special area, such as music, 
which might not be essential to the .model, or whether it 
covers broad approaches to teaching ;j^nd finally, whether it . 
is consistent or inconsis^t;.etit with the model. If training 
from someone other than the sponsor such as the local Hea^ 
Sta.rt office or an RTO consultant, is inconsistent with that 



sponsor' people are-, how much of this additional input^ i.s m|ide 



*fv«mi ,the sponsor, -for example |^ we would expect implen entation 
tq'^suf f^er. ' ' • . * r . 

>^ Conclusions on<. Sponsor Input: 

. ^ From the dai»a ayailable on "pre-service and in-service 

training/ .and" on limited aspects. cf continuing support and ' 

■ > ■ ■'■ 

feedback to the teachers / it appears. - that sponso r input 

va ries among mo dels , and among s.ites wA thin mo de Is. More- 
% over/ there are^fewer PV-NPV differe nce s- "than di fferences 
•^am' onq site's and models. Many of the differences which do 

exist are interactions that indicate that /Ufferent models, 
^nd sometimes different sites within the same models show 
, different relati&rrfships between experimental and control 

classes . 

These findings support the argument that implementation % 

V 

is a^more complex process than had been imagined originally. 

We have discussed the need for a broader definition of in- 

put ip explaiYiing variation in levels of implementatipn in 

terms of both an expanded view of sponsor input and a recog- 

- nition of non-spon so r inputs. While we emphasize the import- 

, ance of these* addi ti^onal areas in an adequate study of im- 

plementation-/ we beiieve t!hat they are not the oply factors 

* which should be 'consi'dereS . We have argued that the process 

o.f impleipentation is interactive, with sponsor input being 

only orie aspect. We turh^ now to the second category of 

" , ■ ^ ?/■ ••• t.. - , 

factor^ v/hich' may affect model implementation. . ' ' . 



LOCAL STAFF REACTION AND INPtfT 

. ^ ^ 

The variables to be presented 'in this section are 

" <\ ' 

•intended "do \ roviAe information on the personal background 
and professional experience of the local teaching staff- in 
Planned Variatiqn. From, one^ point of view, we are interested 
in these variables descriptively^ simply for whar. they tell 
us about \he types olf teachers and aides working with the 
models and whether the. Planned Variation staffs are differ 



ent from non-Planned Variation staf^i; 

I 



V 



From another point of view, we are ^.interested in 
exploring these variables in' terms of whether they* explain 
variation in level! of implementation. We have ass^rtec^ tna* 
irnplementation is' an interactive process which is influenced 
by the local staff as well as by the sponsor. We expect 
that .nodel input will not be effective unless _the . model is 
adopted and used by the staff. The determination of whether 

.V 

the model is adopted depehcis in' part on staff input. The 
skills, preferences/ and previous teaching methods of. the 
local staff may affect levels of implementation. In part, 
adoption of the model also depends on the staff's reaction 
: to it. diking the model and the people on the sponsor's » 
staff, and the;cefore being willing to make the effort of 
trying to work with-^ the new program will undoubt^edly affect 
implementation . 



V 
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Our expectations abovvt the relationship between staff 
'factors and implementation also assume that the models 
repr.fsent new ways of teaching \for the majority of teachers 
in Planned Variation, 'As a result, implementationN^ill be 
a diffi'cult process because of lack, of mastery artd knowledge 
of the new skills and techniques necessary for full 
implementation) and, in some cases, because of the need , to 
break old habits. The Consultant Re.pcJr.ts support this 
assumption. One consulttant, for example, states vYtat 
"...the pull of the traditional seems to wipe out tjie new 
model," And ^another concludes her discussion. on the 
di flhi^ulties of wo^rking with a new program with an observation 
"... I could not i)ut wonder, what is -'the tolL on teachers 

of this model?" Thus, it seems reasonable to assume that 
implementing a new model is difficult. At the same time, 
oii^ hypothesis about the importance of previous skills and 
preferenpes suggests that implementation will be less 
difficult for some t^eachers than for others. As with the 
sponsor input, we find th\at this category has many dimensions, 
^ which do not all influence implementation in the* same way. 



In order to examine the 'staff background variables from 
both viewpoints, we will first present d^ta to provide a 
description of the staff in this study. Second, we v;ill 
examine the same variablls as determinants of levels of 
implemei^itation and v;ill suggest other areas of study. In 




The data presented in Tables 20/ 21 and 22 follow the 

organization of the previous sectV>ns. Table 20 summarizes 

i ' 

/the unweighted means analyses of variance for , the, standard 

balanced design of PV classes only; significance levels and 

allocations .of variance are given. Table 21 shows the 

analyses for the comparison of Planned Variation and control 

classes, again using balanced designs but in this case 

involving only 6 models and 12 sites.. It should be remen±)ered 

^that the model and site effects in this table ore based on 

PV and NPV classes combined, and therefore are difficult to 

interpret, '^'fable 22 shows the means for the models included 

in the analyses of the'PV classes. For variables with 

si.gnif leant model di f f^rGnc;es , the extreme means are ' ' 

r 

underlined'. The means and standard deviations for a^ll sites 
and models on these variaJDl>es can be found in Appendix. D, 
Table 4./ • , . 

We will not examine tlje specific variables contained in 
these tables in detail as we did in the previous sections, 
because they are not directly affected by the experimental 
treatment. The major impression to be drawn from the tables 
is that there are large differences among models and sites 
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TABLE 20 

Staff Background 

Analyses of Variance for Planned Variati bti Classed 



) 



Variable Name 



Sources of Variance: 
\ 

Site Class 
Within Within 
Model Model Site 



. Teacher : ' - . . 

TQ#33 Live in the neighbor- 21.58?; 
hood where most of the - ***- 



children live 



^17.20% 



61. 22^ 
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on staff background characteristics, and relati.vely small 

35 • 

PV-NPV differences. . • 



A brief consideration of two variables is useful in 
dlinonst rating hov; the data can be 'interpreted. For the' 
teacher certification variable (TQ#41) , there Vi^e strong 
effects for both sites and models. The model/ means 
range ,from Far West , where 77% of the teachers have their 
teaching certificates, to Kansas where none of th^- teachers 
?3q. Arizona hasf^the highest variability within models with 
all teachers in ' Lincoln having their teaching cferti f icates , 
and with none of the teachers in Lakewood having,;.theirs . 
There is a moderate PV-NPV effect for this variable, which 
is; reflected in the finding that , , ov>jrall , 60% of the non- 
Planned Variation teachers are certified , while only 38% of 
the Planned Variation teachers are. There , is also* a 
moderate, site within model interaction with._PV-NPV, which 
indicates that the relationship between PV and NPV^ teachers 



^^Th|.s finding does not hold for. the teacher aides. Analyses 
of tr^eir questionnaire responses reveal significant effects 
only for level of education; age, years of Head Start exper- 
ience^, ethnic background, and^place of res,idence (whether aide 
lives in the same neighborhood as her students) show no 
significant differences among sites and models or between 
Py-NPV. In addition, some teacher variables also show no 
"significant differences. Nursery school practice teaching 
and kindergarten or first grade practice teaching are two 
(TQ#40) ; overall,, few teachers checked these items. In 
addition, analyses of types of experience other than in 
Head Start (TQ#42) could not be completed because very few * 
teachers in the total sample checked these items (34 of 264). 
There are. not enough responses 'to draw conclusions other 
than a general one that very few PV teachers have types of 
teaching experience other than Head Start. 
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is not the same in a^l .sites. For the. second variable, 
nuTtiber of years experience v;ith Head Start (TQ#42) there 
are moderately significant model effects. The means. show 
that teachers in Bank Street have, on the average, . taught 
for 2.6 years and teachers in the Florida models fiave taught 
for^ 2.5 years. Both .of these means are lower than those 
for the other^ models which range from 3 to 4 years average 
experience. The site effects are stronger than model effects 
on this varii^le.. Arizona and High-'Scope have the 'largest 
variation. In the Arizona mode 1^ the me ^ins . range frqm an 
average of 1.6 years of experience in L incoln to 4 . 6 years 
in LaFayette , and in High Scope the range is from 2 . 2 years 
of experience in Fort Walton Beach to 5 . 2 years in Seattle. 
There are no differences between Planned Variation and 

control teachr^rs on this dimension. 

These and the other findings for the staff background 
variable^, then, indicate that there are wide variations^ 
among teachers in Planned Vajriation in their teaching . 
experience and ' training, their level of education, their 
age, place of residence, and assignment to PV. Thus, it 
. appears that the teachers with whom a sponsor works .,in one 
site are^ 'different from those in another site. Koreover, 
the situations which one sponsor faces are different from 
those which another sponsor faces. FinajMy, the relative 
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lack of Py-NPV effects sxiggest that on many dim ensions , 

/■ • * 

the differences among teachq^rs are not unique to Planned 
♦ — — — ■ ■ — I ^ ^ 

Variation. " . , 



Staff Reaction and Input as. Determinants of Levels of 
Implementation : ' 

The finding of variati^/ons in staff .background characte;r- 
istics supports the expectation that these variables might 
be useful in explaining variation in levels of implementa- 
tion. In considering them in this framework we shall 
follow the division introduced earlier, between staff input 

and reaction to the model. 

By staff input we mean the characteristics and experiences 

which an individual brings to the situation in which he or - 

she is to implement a model program. The data just presented 

fit into this category . _ The— assxunp t-loTL- 1 h at- iit^ lejnenfcing 

a model is a difficult process is important here. We have 

asserted that full implementation is difficult because it 

requires knowledge and mastery of new skills and techniques 

in teaching. We also asserted that implementation will be 

more difficult for some teachers than for others. The 

consideration of sta'f:^input variables provides a. basis for 

making predictions about which teachers fall into which 

group. ^ 

The pi-edicted relationship with implementation is 

clear for the number of years in Planned Variation variably: 

'\ ' . - 

the longer a tieacher has. worked with^ a model, the more 

■ - \ ■ \ 



-128- 

successful she should be at implementing it. 36. por other 
variables , the relationship with levels of implementation 
is less obvious. In general, we might predict that a young, 
inexperienced teacher would tend to implemen.t a mpdel to a 
greater extent than would an older, more experienced teacher. . 
This prediction assumes that an older teacher has an 
established] pattern of teaching. For her, implementing a 
new model involves not only learning new skills, but I'reaking 
old habits as well. Particularly if she is comfortable with 
her established methods, implementing a new model may be 
very di^:ficult. In contrast, a new teacher might be very 
receptive to a model because it provides an organized 
fraraework within which to cope with her new duties. 

For trie impact of level of education, additional training 
and certification on implementation, we can make Qpposing 
predictions. On th6 one hand, more education and more 
training may be detrimental to model implementation because 
.a- well-educated teacher may be aware of a rang'e of alternative 
approaches to teaching young children, and may consider the 
model simply as another alternative. In such a case, she 
might ignore the model, or pick only parts- of it to work with, 

•^"Unfortunately, this prediction is not borne out when the 
sponsor ratings are used as measures of implementation . As 
we pointed out in Chapter 2, however, it is unclear whether 
the finding. i^s due to a real lack of differences between 
first and^second year teachers or whether it is an artifact 
of the instrument. 
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\ 

' ' ' t 

rather than trying to implement the entire model; On the 

other hand, a more highly educated teacher might be familiar 

"wit^h the concepts a sponsor is trying to introduce through 

his model, and therefore might understand the mo,del better 

than a person who has less training. In this case, a higher 

level of education might be beneficial to implementation. 

We pr6pose that the determination of which prediction is 

realized depends on the interaction of education and training 

with other variables su^h as reaction to the model and 

previous methods of teaching. We will return to these ot^er 

variables shortly. - . 

Race and place of residence of the teacher are teacher 

characteristics which are of interest in describing Planned 

Variation teachers, but which are difficult to relate to _ * 

implementation as input -variables . Place of residence is better 

considered as it relates to a total site, and. will be considered 

as a conte^ft variable. 

0 

The second category of variables through which the locai 
staff interacts with implementation can be called reaction to 
the model. We expect that if a teacher likes a model and the 
people who sponsor it, she will have a higher level of 
implementation than if she does not lik^ it^ 

One aspect of a teacher's reaction to the model is 
her reaction to the sponsoV staff and to the trarning they 
give. A vari'able |)resented in the sponsor input section 



r 
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under continuing support .and feedback is relevant here: 
satisfaction with training, "^"^ It is possible that high 
satisfaction with training indicates a positive reaction 
which may be associated with greater attempts to implement 
the model. The more important side of this dimension, 
however, is probably the n^ative one. We would predict 
that if training is very poor that people would have little 
motivation — or skills — to implement the models. Similarly, 
if the local staff views the sponsor'^ staff -as unprepared 
or incompetent, we would. expect implementation to suffer. 
We have no data on this latter variable, . ' 

"There are several other variables for which there are no 
data, but which are crucial to understanding the relation of 
staff characteristic to levels of implementation. One of 
these is reaction to the content of a model, or to the model 
as an approach for teaching. This aspect of a teacher's 
reaction is probably more important than her reaction to the 
sponsor's staff and training. We expect that liking £or a 
irtcdel is determined primarily by the ■ teacher 's philosophy 
of education. If the model is consistent with her views . 
about the nature of children and their learning processes, 

3'^tfe^^ould be remembered that this variable asks about i 
training in general, and can only used- as a rough 
estimate of satisfaction with m^del training. 
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then the teacher will "be more apt to like the mciel and 

38 

will make a -greater attempt to implement it, 

AnQthej;; 'v^ry important variable foifi?whigh there are no 
d^ta is the method of instruction.'used by a teacher before 
she bag^ working with -a model.. In one sense> this variable 
fits v^ith the above discussion about the - consistenay of 
a model and a teacher's philosophy. We would predict that, 
a teacher whose previous style, is consistent with that 
required by the moc^ will react more favorably to the model 
than will a teacher whose previous methods are very, dif-ferent . 
from those of the model. ' In another s^nse , this variable 
can be seen as input and is directly related to the assumption 
that implementation is more difficult for some teachers than 
for others. We predict that implementation will be easi^-^r ^ 
if the model is consistent with a teacher's previous way of 
operating a*class and relating to cfjildren , because it will 
priinarily depend on learning new skills' rather than breaking 
old habits. For example, we would predict that a teacher 



38 ' ' * 

There are two variables, whether the teacher volunteered . 

or was assigned, first to the center and then to Planned 
Variation, which inight be used as indirect measures of 
liking. for a model. We might, predict that if a teacher Has — 
chosen to work with t^e model, she may like it better than' 
if she had been" assigned to work .with it. Choice indicates 
some preference for a model, even before beginning. /assign- » 
ment without choice does not involve any feeling of prefer- 
ence, and in some qases , may even involve forcing a* teacher -.• 
to work^with a program she does not like. This somewhat 
tortured reasoning, however-,- does not substitute for a 
direct v^question of whether a tieacher likes a model. 
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who has been accustomed .to working with children in structured 

academic lessons would find it easier to work with the 

Oregon nociel than would a teacher who gives the 

children a great deal of time for free play and individual 

exploration. 

v., 

to which the model is- understood, a filial 
.ch there are no data, is an important 

a teacher cannot implement a model when 
fer^tand' what is expected of her, or how she 
r the • requirements of the model. ^ 
In surt^ary , ' it appears that staff variables may be 
intporbant in> determining th"^ difficulty and motivation for 
implemeniiing a model . We have sug^sted that levels of 
implementation may "^e partially deterlnined by factors such 
as the teacher's level of edupatio^, experience in Head Start, 
previous methods of teaching, a;B will as reactions td model 
training and the content of a Tn<^el. We would expeo^ , . then , 
variation^ in staff fectors raighfc explain variation i^ these 
levels. Although the variables plfes'SntrScJvjn this seiction tap 
several dimensibns of staff input and reactionV^the/re a;re no 
data for ottier crucial diiuensions* For a complete un'Ser-- 
standing of the relation betjween staff (Characteristics and 
model implemen.tation, we need information on the teacher's ^ 



\ 



ERIC 



-142- 



liking for and understanding of the model, and her previous 
methods of teaching. " 

' Implications for Study Design : 

The fact, that we expect teacher background variables to 
influence implementation has implications for* the design of 
^he Planned Variation study. It is impdrtant'to comment on 
these before continuing our discussion' of factors wh.ich 
influence implementation. Ideally, in an experiment / one 
assumes that the situations in which the treatments are to 
be tested do not differ sy temati,cally except in ways caused 
by the treatments. This assumption is ^important in drawing 

J'conclusions about * outcomes of the treatments • The finding 
of significant differences, among sites and models on teacher 
background characteristics suggests, however, that this • 
assumption is not met: teachers differ systematically in 
ways which are act caused by the treatment. ^ 1 f 



-^^This section focused almost exclusively on the relationshi 
of ^ach*er characteristics to implementation. Teacher aides 
were not included primarily because of the limited data 
available cfn them and because of the lack of model or site 
differences in the data which do exist. We expect, however, 
that in classes where the aide is included as a teaching 
^ partner that many of ' the same relationships will* hold. "The 
whole question of the teaching aide's role in implementation 
defeerves further study. - 

• 40 

This is another aspect of an assmnption in Chapter 1: 
that treatment is a primary determinant of what happens 
in the classroom. " ' , 



The differences among sites on these dimensions are 
not hard to understand:* all teachers within a, site are 
^^^^^^rected by the same 'adminis trators , and administrative 
policies in Head Start vary from one center to. the next. 
'Sites may place^^i f ferent emphasis, for example,, on having 
community member^ on the teaching staff, on requirements 
for certification, or on additional training. Different' 
hiring policies or career ladders, then, may result in 
systematic differences among teachers in different sites. 

The systematic differences among models on these' 
variables are. more difficult to deal with. Even with large 
differences among sites, we did not expect systematic model 
differences. A sfearcii for. an explanation for these * 
differences in such factors as regional differences among 
models reveals no pat-tern which relates to teacher • 
characteristics . The finding that not only does each 
sponsor face a variety of situations in tryinc^ to implement ^ 
his model, but that the situ ati ons differ from mode l to- 
model, confounds the treatments \n Planned Variation, It 
will be. difficult to tell whether differences in child 
outcomes result from model differences or teacher differences 
This is a serious problem with the study design. 



I ■ ; 
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OPERATIONAL (CONTEXT OF THE STAFF 

The third category- of factors relevant to 
iinplementation is the contej^t in which th'e staff .operates 
the sponsor and staff inputs do not operate in. a vacuum, 
but rather interact with the situation around them.-^ For 
example, it is probable that implementation will be difficult 
when the sit'3 is in ^urittoil over the loss of funding, even - 
if sponsor support to a site is strong, and the teachers like 
the model. Three ^spects of the operational context can be 
identified. In suggesting how context variables ma^ be 
related to levels of implementation, each aspect will be 
considered separatalv. in the presentation of available data, 
however, the tiiree dimensions'^ will be combined because the data 
are inadequate for separate discussions. 

The first dimension of the operational context, which 
we shall call site characteristics, includes variables which 
help to determine what a site is like. For the most part, 
it includes straightfprwarci and easily obtained information 
such as the size of site or adequacy of facilities. In part, 
however, it also includes less tangible factors such .as 
the aunosphere in , the ^ centers and the values on which they ' 
operate. The second area, site administration, includes ^ 
those factors which determine how smoothly, or efficiently, 
a site operates. Our assumption is 'that if the site is not 
functioning well, the ene,rgies of the staff will diverted 



to administrative problems "^nd away from model implementation 
Thus, efficient site management may be a minimal condition 
for implementation. The third category, administrative 
Support for the model, is based on the assumption that the 
Head Start director is a key person in the implementation 
process. We predict that the director's opini6n about the 
model and about PlaniTec^ Variation will influence implementa- 
• ti^n by affecting the efforts of the teaching staff to 
work with the model, and by determining the administrative 
arrangements made jn relation to the experiment.. 

Data : * ' • 

Three items from the Teacher Questionnaire are, relevant 
to^ a- discussion of the context of implementation'. Table^ 23 
shows the analyses of* variance for these items for both^PV 
classes only and for PV-NPV comparisons. The analyses fo^r 
PV only are based on the standard balanced design. Those 
for the PV-NPV comparisons are based on the smaller sam.ple 
of 6 models and 12 sites. Table 24 gives the mec-^ns on the 
three variables for the models included in the analyses. 

TabJ.e-2 3 sHows that th^re are moderate differences 
among models on the * teachers ' satisfaction with their 
working cpnditions (TQ#24) , on th^^^xc^nt of parent invol-ve- 
ment in the classroom (TQ#29) , and i'lT^the frequency of 
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TABLE 23 

r 

Context Analyses of Variance 





Planned Variation 


Classes 


Only: 








Variable Name 


- ■ ■ . 
Model 


Site 
Within 
, * Model 


. / 

Class 
Within 
Site 


df Class 


TQ#24 


Satisfaction with working 
. conditions 


9.36 t 
* 


19.12 


71.52 


134. 


TQ#29 

» 


Degree of parent involve- 
ment 


12.18 


15.57 

NS 


• 72.31 


126 


TQ#30 


Activities for parents 


9'.38 
* 


22,00 
** 


68.61 


129 



.Note: See Table 8 for a description of these analyses. 



\ 



PV-NPV Comoarisons 





Variab.le Name PV-NPV 


Model 


Site . 


Model 
.X 

PV-NPV 


Site 
X 

PV-NPV 


■, d'f 
Class Class 


TQ#24 


Satisfaction with 0.2' 
working conditions NS 


4.96 

• NS\ 


10.77 
NS 


5.5-2 
NS 


9.31 

NS 


69.16 
57.10 


TQ#29 


Degree of parent 0.20 
involvement NS 


18.11 
**,* 


5.79 

NS 


M.81 
NS 


13.99 


TQ#30 


Activities for ^ 0:00 
parents NS 


JLd.03 
** 


9.71 

NS 


3.99 
NS 


7.16 


63.11 


Note: 


See Table 9 for a description of 


these 


analyse 


s .. 
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TABLE 25 
Context 

Means and Standard DeviatTons from Sponsor Site Assessiment 





t 


Far West 


Bank St. 


Oregon 


Kansas 


Florida 


a. 


Turnover rate .teachers 


1.5, 
.56 


i.8 
1.30 


> 

3.0 
1.0 


2.3 
. 1.89 


3.8 
1.64 


b. 


Turnover rate aides 


2.2 
1.30 


l.'O 
■ 0.0 ' 


3.5 , 
1.50 


2.3 
.94 


1.8 
1.30 


c. ^rnover ^ate children 


3.5 
.87 


1.8 
.83 


2.0 
0.0 


1.7 
.94 


3.0 
0.0 


d. Intra-staff friction 


2.5 , 
1.12 < 


/" 4.0 

[ ■ .71 


2.0 
0.0 


2.3 
.94 


2.8 
1.30 


e. 


Regular attendance - * 
teacher s 


4.2 
.43 


4.2 
1.30 • 


2.0 
0.0 


2.3 
.94 


0.0 
0.0 


f . 


Punctuality ot teachers 


4.2 
.43 


'4.2 
1.30 


2.5 
1.50 


0.0 
^ .00 


0.0 
.00 


g. 


Regular attendance 
children 


3.0 
' .71 


4.0 
1.22 


3.5 
.50 


3.7 
.94 


3.0 
.00 


h. 


Suppoft of iQcal Head 
Start 


4.2 
. .83 


3.5 

. .87 


3.0 
1.00 


3.7 

.94" 


2.7 
1.09 


i . 


Support of the community 


4.0. 
1.22 


,1.22- 


4.5 
.50 


3.7 
.94 


4.0 
. .7-1 


j- 


Support of PAC 


4.5 
.50 


3.0' 
1.58 


4.5 
.50 


3.7 
.94 


3.8 
.43 


k. 


Adequacy of: plant in- 
doors 


4.0 
.71 


3.2 
.43 


2.5 

1.50 


3.0 
1.63 


1.7 
.47 




Adequacy of plant out-- 
doors 


2.8 
1.22 


3.0 
1.78 


2.5 
• .71 


2.3 
1.50 


1.7 
.94 


n. 


Rapport between adminis- 
trator and staff 


3.5 
.50 


3.5 
, .50 


4.0 
.00 


3.6 
1.63 


2.2 
.83 


o.' 


Rapport between staff and 
children 

• 


4.2 
.43 


4.0 
.71 


4.5 
.50 


3.0 ' 
.00 


3.5 
.87 


p. 


Rapporf between sponsor 
and local Head, Start staff 


4.2 
.43 


4.0 

.71 


4.0 
1.00 


4.3 
■ .94 


3.5 

1.12 
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Only means for models in the analyses of variance 
are included here* There were no significant model 
effects on any variances. The standard deviations 
give an indication of the variation among sites. 
The Enabler^ model is'-not included here because the 
Enabler consultants- were not asked to complete the^ 
Sponsor Implementation Report, 
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* . ' • 

. . ' 41 

activities of fered for parents (TQ#40) . Examination of 

•>» 

the means indicates that the teachers in Far West ( 2 >Q ) and 

... ' ~ 

EDC ( 2 , 1 ) tend* to be the most satisfied .with their working 
conditions while the teachers in Florida -( 2,7 ) are somev/hat 
less satisfied. Oregon has the highest level of parent 
involvement ..{3.4) and Kansas tends to have the most freauent 
parent a,ctivities, averaging around once a month ( 2.1 ) , 
although all models are fairly similar on this variable. 
There , are also significant. diffe'rences among sites within 
mo<3els on satisfaction with working conditions and activities 
for parents. Bushell has the largest variation on both 
variables: the teachers in Mounds are satisfied with their 



working conditions ( 3^. 8 ) wliile the te^achers in Portageville 

have .mixed feelings ( 3.2 ) . Activities for pai^ents, are 

offered weekly in Mou)ids (1>0) , but only , twice a year in 

. ^ 

Oraibi ( 3.1 ). Finally, there is only one. PV-NPV effect for 
-these .variables: an interacbion with eites on the amount 
of parent involvement. This- indicates that the relationship 
between PV and NPV varies across sites. 



41 . 

In analyzing each of these variables, the form was altered 

slightly from the original. For Wording conditions and ^ 

activities for parents, an average was taken across the 

specific, categories of conditions and activities. For the 

parent involvement item, the, transformation was slightly 

different: the frequency of parent involvement (every day= 

5,^ once a week=4...) was weighted by the number of parents 

in that category (e.g., 6 parents worked once a week = 6 X ^) , 

summed across the categories and divided by the total number 

of parents involved. The resulting scores are a combination- 

of frequency of participation cind number of parents , where 

a high score indicates a greater dnvolyement . The original 

format of the questions are found in Appendix B. 



4 
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The primary sources of data for this area, however, 



are the site assessments from the Final Consultant Report 

and the Sponsor Implementation Reports rather than the^ 

Teacher Questionnaire.* The means and standard deviations 

for the variables from these si'Lj assessments are given in 

Tables 25 and 26. The analyses of variance for these reports* 

differ from those for the Teacher Questionnaire because for 

each variable, there is only one observation per site. As 

^a res-ult, we can only* test for differences among models and 

not for differetices among sites. Because o^ missing data and 

single site models, only 5 models were included in the analyses. 

The analyses of the site assessments reveal no significant 

differences among models on the items relevant to the context 

of implementation. It is not clear, however, whether these 

findings result from a real/ lack of • differences , or from the 

low power of the test Although site differences cannot be 

tested statistically, the large standard deviations on some 

• 42% 

of the variaJDle^ suggest that sizable dif fe-rences- do exist. 1» 
Thus ^ we might tentatively conx:lude that there are differences 
amona sites within"* models on some of the items in the site 
assessments. v " ' 



42 ' 

In these reports, where the scores can range from 1 to 5, 

a standard deviation of 1.00, for example, results , when ' 

the site scores are 1, 2, and 3'. A standard deviation of 

1»73 indicates a wider range of sC:o^es such as 5 , 4, and 1, 

while- a value of .58 indicates a narrower range such as 1, 

1 , and 2 . ^ 




It should also be 'fioted that the degree of eorrespondence 
between the sponsor ^d - qonsultanC ratings .is not always high 
Table 4 in Appendix fr-^^ows t^ correlations between the^ 
two sources for each item. The . correlations' range^ from a 
.10 for the turnover r,ate of children to .6 8 for adequacy of 
the physical plant inside-; — ^S^ce the items were essentially ■ 
identical^fo'r the two groups of Veople^ making the assessments, 
the low correlations indicate thAt the ratings are unreliable: 

far one site, the. ratings vary according, to Who is making ' 

*>■ ■ ••. . ♦ • 

them. It may oe i:hat the items are so gen/eral that they c^n 
be interpreted in several w.ays . In any case, it 'is cleai: 
that the* findings from £he site assessments -should only be 
used as general indicators of the^, context in which implemen-- 
tation -is baking placre. ' ^ 

^^ithin . these-- limitationis , we feel' tha^ the assessments 
provide interesting descriptions' of the site^. Since it is 
difficult, however,- to integrat"^^0, variables into a 
coherent picture, ^/e performed a factor analysis on the 

%■ 

cons ul*t ant site assessment in an" attempt to reduce the 
variables to a manageable number of dimefnjions. The consul- 
tants' rating rathe r\th^n the sponsors' were used because 
theraaromore d^ta fc/r them. The results of the factor 
analysis are included as Appendix E at the end, of tl 3 report. 
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L few general trends are apparent from the assessments. 
From the consultant reports, it appears that all models 
tend toward the lower end of th^ scale on friction, while 
they. tend toward the high end on punctuality, regular 
attendance, support of Head Start personnel and PAC for the 
model, and rapport between the staff and the children • From 
the sponsor reports,, all models tend toward the high end of ^ 
the scale on regular attendance of the- chi Idren , 'support of 
PAC and the community for the staff and the children^ but all 
tend toward the low end on the adequacy -^f the physical plant 
outdoors . 

We^can conclude, then, from the data \jhich exists for 
the context in which implementation takes place, that there 
are differences among sites on a number of dimensions. The 
siqnficance of those differences, however, eannot be tested 
for the site assessment data. The di f ferences amon/a models 
are significant only for three items from the Teacher 

V. 

QuestionHr^ire and there are low correlations between the two 
sources of data for the site assessment's^ (j 

Context Vari-ables as Determinants of Levels of ,Ilnplementation 
Site characteristics : . The first category of coni^ext variable 
which may be related to implementation revolves around the 
broad question of what the sites are like. We expect 
different dimensions of this category to interact with 
implementation in different v;ays . One characteristic of 
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the site, and tjie . community of which it is a part, is 
directly related to model implementation: suppoit for the 
model. There are three items on the site assessment vvhich 
report support from the lo^al Head Start staff, the PAC, and 
the community for the model. 

Other' site characteristics are less explicitly related 

-V. 

to implementation, but are no less important. The satisfaction 
with working conditions and adequacy of physical plant ' 
variable? are relevant here.^ expect that the conditions 

under which teachers must work are important primairi ly^. in 
|:erms of est'ablishing a minimum base for implementation. 
Above the minimum Ipvel, we do not expect this factor to have 
an appreciable influence on implementation.^ Below the minimum, 

4 

however, poor working conditions will make implementation very 
difficult. Facilities would seem to be a particularly 
important element of this factor. If for example, several 
classes are housed in one room, the lighting is poor, the 
noise level is high, or the supplies are inadequate, we 
.would expect model implementation to be hindered. 

Two additional variables for which there are data^ . 
parent involvement and activities for parents, may also be 
related to implementation. We expect that a center which 
attempts to involve parents in clas^sroom and other Head 
Start activities will provide a different context for 
implementation than one in .which parents are not deeply 

1 • • 



ERIC 



V 



-157- 



involved. These variables may be part of a larger dimension" 
wnich contrasts a community-oriented site with a school- 
oriented site. A community-oriented site is one which 
encourages full parent participation as opposed to a' school- 
oriented site which maintains more formal, distant relation- 
ships with the parents. If thp.s dimension is valid, it 
might also be reflected in other factors; such factors might 
be hiring priorities, as measured by • the number of teachers 

r 

who li v^e in the same neighborhood with the children, or the 

number of teachers who are cert>4fied (Both of these variables 

If- 

were introduced as staff iDackground factors). Wa would 
expect*^ that a community-oriented site would emphasize Ifiring 
neighborhood pqople, while a school-oriented ' si te would 
place a hj-yher \ralue on credentialled teachers. it seems 
unlikely that this dimension would, have the same relation 
lo model implementation in all situations. It may be that 
it interacts with models: some models , may be better for 
community-oriented *sites while others are better far sites 
With a school orientation. Mo^e study is needed on this 
aspect of the site chal^acteristics ; the present variables 
are not adequate for exploring it, 

>»■ 

Another variable which in some ways is an extension 
of this school -community dimension, is the delegate agency. 
This is a question of who runs Head Start. It is a 
continuation of the previous discussion in the sense that 
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in many cases Head Start is run either by CAP, the Community 

Action Program, or by the public schools. If we are 

concerned with defining the context a site provides through 

its values or orientations, the school-cofrununity contrast 
\ ^ '43 

- applies fairly well, and might provide a tool for increasing*^ 
'our understanding of the implementation process. In anoth^ 
sense, howeyer, the variable is not an extension of the 
previous discussion. The delegate agency variable may be 
■ related to implementation in more concrete ways than values. 

It is possible- that public schools provide contexts 

/ ■ 

which facilitate implementation because they- may have more 
'Materials, better facilities and more efficient operating 

^ procedures than other .agencies which administer Head Start. 
This is another site characteristic which deserves exploration. 

A third aspect of* the'N^elegate agency variable which 
may be related to implementation * is the presence of day care. 
We would predict that model implementation will be more 
difficult in a Head Start center which is combined with a 

\ day care program than i:i one which is operated independently* 



We base this prediction on the assumption that teachers and 

V * 

children both terrd to be tired from the longer day involved 
/ ■ 



43 



One problem with matching a delegate' agency l^fl^l with an 
orientation label i^s**, for example, that there can school- 
oriented cenC^rs whi-ch are not administered by ^he publi^c^ 
schools. Impjiementation wi^^l- be more difficult than in 
centers where there is less strain. - ' ' 



with c^y care. As a resi^lt impleifientation will be more 
difficult t^an in centers where there is less strain. 

> These variables do not exhaust the aspects of site 
cliaracteris tics which may be rjelated to implementation* 
There are no doubt still more areas for which we have no 

information, such as the size of the site and location of 

-I 

the centers. In relation to the size variable, we would 
predict that if a site is very large and Planned Variation 
is only a small part of the total operation, then Planned 
Variation may receive less attention than in a site wltcre 
it is a major c^cem. As ^ result of being less important, 
model ' implementation may suffer. -Similarly^ the location of 
classes withj^n the site may influence implementation. We 
would -predict that if Planned Variation classes are spre^ 
out among centers rather . than being used together, • the 
isolation of .teachers trying to work with a new model may 
hinder implementation. 

Thus, t^ere are a' ,rlumber of site characteristics which 
may i^e related to implementation. Undoubtedly, we have aot 
identified all relevant characteristics, but we' have raised 
some important issues. While thfere is no' evidence of 
variation in some of the variables considered here, thfere 
are differences amona sites on o^ers^^ hiring priorities, 
parent involvement, and delegate ^gerlcies . ^le existence of 
•differences among sites on these variables , then, supports 
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the hypothesis that site characteristics may part-" ally / . 
account for differences in model implementation. 

Si te admin is tration ; Site management, a second dimension of 
context which we would expect to be related to implementation, 
is primarily a question of how smoothly, or efficiently/' 
the site is managed. If a center is functioning v;ell, then 
the teaching staff wi.thin it can concentrate on the classrooms 
and .on implementing the model. These variables are seen as > 

. ■ V 

minimal conditions for implementation: they alone do not 
guarantee implementation, but they, do support it* • 

A number of variables from the site assessment can be 
considered as indicates of good management. One component 
is stability of the center, particularly of the staff. This 
includes elements such as turnover rates, punctuality, and 
regular attendance. These variables are found in the site 
assessments. We predict chat implementation will be at a 
lower tevel if staff members are frequently late or'abseht,^ 
or if they work at the centers for only short periods of 
time^, than' if the staff are on the job regularly. ""tJur 
assumption is that training ne.^ teachers, finding substitutes, 
and covering extra classes diverts energy away frtom model., 
implementation. / 

Similarly, we predict th^^t friction among the staff, 
a second compo^ient of site management, diverts ^resources 

from implementation.- The frict^ion variable froj!^ the site ' 

. \ I 
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as se s s me nt-vi provides data which is relevant here. If there 
is disagreement about such concerns as role definitions, we 
would expect conflict and neglected duties. Such a 
situation may hinder implementation. On the other hand, 
if all staff members know ifeheir areas of responsibility and ^ 
thQ3e of the people .around them, this concept will "not be an 
issufe in implementation. » . ' 

In addition to these aspects, there are other components 
or site administration for which we have no data bu"^- v/hich 
may Influence model implementation and therefore sliouid be 
considerec^ One is adrnini^trati ve cqmpetence. Again, the 
negative end of the dimension is the important one in this 



context. We would expect that unless the a(iministrative - 
staff is at least minimally competent, — that is, unless 
^the^y can manage su;rh tasks as getting the children to the 
schoo^i^nd fed^ paying bills and salaries, and getting the 
buildings open and heated- — model implementation ^^^not 
proceed. 

In contrast to the previous variables' which refer to 
- « . ♦ 

internal issues, funding security involves the concept of 

' efficient relations with people aibd agencies outside the 

Head Start center. "^Clearly; funding problems hinder 

implementation when they prevent a site 'from hiring enough 

people to work v/ith the model. Even when staff members do 

the threat of 



not actually ) If ''^e their jo^,l howevcv^. 
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budget cuts ^ late pay, and general funding instability may 
iinterfere with implementation: if staft Jdembers are beset ^ 
with worrying about whether or not they will have a j^b 
next month , 'working to avoid this crisis will probably divert* 
attention from the classroom. There are no data /for this 
item either. 

The same relation to implementation, t|ien, holds for 
« •' ' ' — " 

all the varicibles which have been introduced as site 
administration factors: problems in administering the site 
divert attention and^energy from fnodel implementation. .WhiVe 
some of these variables may be more important to implementa- 
tion than others, we would, expect , in general, that they 
will be additive: the more • problems a center has ^ its 

operations and relations with other groups, the less attention 

' ' ' \ ' \ ' 

il; as a result, implementation will 



will be given to the mode! 
suffer. 

Administrative support for the modele r V^e expect that^all 
t . 

Head/ Start administrators will have opinions about the models 
with which theii;^, teaching staff must work, and about the^ 
importance of the Planned Variation experiment. Further, 

4 4 " / . 

A distinction jbetween opinipns about the model and opinions 

about the Planned "Variatioi;r experiment is^ necessary because- 
the former alpne will not petermine implementation ; The 
Planned Variation design , involves the assumption that the 
models v;ill be replicated in- the sites. Even if a director * 
likes a model, she may interfere *v;ith implementation if 
she does not. understand ancl respecj/ the experimental design 
of which it is a part,. 
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we expect these opinions to affect -model ^ implementation . • 
in at least^^two ways: by affec1:ing the attitudes of the 
teachers toward the model, and through administrative 
decisions which facilitate or hinder implementation, Altnough 
there are no systematic data available through which to test 
these hypotheses, ane'cdotal evidence suggests the issues; ✓ 
are at least worth raising. • ' 

Before discussing in more detail th6 ways in which 
administrative support may influence implementation"^, it will 
be helpful to comment briefly on the sources or determinants 
of such support, although '^without data we cannot test our 
hypotheses. This question, like many others, is complex 
and deserves more thorough study than we are able ""O give to 
it.- We, expect, however, three kinds of factors to determine 
an administrator's support f^r a model. The first is the 
adminstrator^'-s^opinion of the sponsor. Since curficular 
models are tied clAsely to the notion of sponsorship in this 
experiment, it seems probable" that the relationship between 
the sponsor and the He tart director will greatly influence 
the latter' s "Opinion of the model and the experiment. In 
evaluating the relationship, we would expect that things such 
as the way the sponsor preserits himself and his program to 
the community initially, ijis continuing attitude Inward the 
sibV and toward the direrHfeor in particular, and^ his 
competenqe in training and monitoring implementation v;ill 
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be important. If/ the director has a high opinloh of the 

sponsor^ we would predict that she would support the model 

to a greater extent than if she did not like him. A second 

probable determinant of an administi^ator ' s opinion, of the 

model is her philosophy of education. We x^ould expect that 

if her values are congruent with those of the model, she' 

will like the model. A third determinant of support can be 

labelled as priorities in operating the program. Jhis 

factor is perhaps less obvious than tffe first two, and is 

better viewed in terms of the administrator's view of the 

importance of Planned Variation than in yterms of liking th^ 

' model. By priorities we mean the director's sense of what 

is important for the program; her perception of the roles 

and goals of Head Start. We expect, then, that the director 

support for the experiment will b^ influenced by her 

priorities: if maintaining an experimental design ViS low 

Apriority, she may intentionally or unintentionally interfere 
J* " 

with implemeffitation by acting on higher priority items. 
Thus, the su^ort an administrator gives t">e model may be 
de-terrained by the interaction of her feelings about the, 
sponsor and his representatives, the c?^ongruence of 'the 
model's approach to education with her own beliefs, and 
the^relative standing of the PV modpl in her syst.^v of 
priorities for the Head Start program." 
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With some feeling about th^ determinants of why an 
adminis-trator woi^ld or would^nolt support the models we return 
to a discussion of how such support then influences 
impl^njerrtation. Unfortunately, we havej no systematic data 
for these questions either, and must rely solely on anecdotes 
fgtr illustrating the ^discussion. To repeat our previous 
hypothesis, we expect that support is- related to implementation 
in at least twp ways: by affecting the . attitudes of the 
teachers toward the mo<feij and by making administrative 
decisions which -.facilitate' or hinder implementation. In 
the" first case, the administrator's opinion is seen as an 

.important determinant of the atmosphere created- witJhin the 

\ ■• ■ • ■ 

site in relation" to the model. We would expect that, in a . 
situation where the director likes the model and thinks 
Planned Variation important, sha will give moral support 
and encouragement to the staff working with the model. On 
the other^^nd, if she does npt think highly \of the model in 
particular, or the experiment in ».^aneral, the c"AtrTiosphere " 

for the staff will be poor^ For example, one £;ponsor 

45 ' ^ 

reports that "Morale problems .were generated because 

paraprofessionals in Planned Variation 'were treated as 

temporary employees by . administration . s It seerus probable 

that staff members in such a situation will be less 

motivated to work with the model than in one where the 

director supports the model- 



45 , 

Tne Sponsor Implementation Reports contain several open- 
ended questions; this quote Is taken from one of them. 
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The sedond way in which adnjinis trati ve support influences 
implementation through decisions and po!|.icies. This 
aspect of support is more coi^licated than the first because 
it includes a larger proportion, and therefore a wider ' 
variety, of the administrator's actions and responsibi*fities . 
We cannot specify all the actions which might affect 
implementation, but we can give examples to illustrate the 
importance of this concept. In some cases, a director will 
not take tjog action necessary to support imple'mentation 
because he or she^does not like thre model. For example, 
one consultant in discussing a number of problem^^ a director 
has with a model and its sponsors » states : "Since he's not 
sure of th6 model, he doesn't want to spend money on materials, 
bocks, etc. that are related to it. With the big financial 
cut, he's worried about resources." 

In other cases, however, the important support factor 
for implementation is^ot liking the model, but respecting \ 
the experimental design of Planned Variation. A director, 
for example, may have a. high opinion ^of the model as an 
apporach to teaching^hildren , but 'unless she understands 
the experimental-design and tries to maintain it, she may 
hinder iiup3 ^mentation bv the "policies she pursues*. The 
relation of such actions to implementation may not be 
immediately— apparent until we remind ourselves of the goals 
of Planned 'Variation. One of the p ;imary objectives of 'this 
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experiment is to test the effectiveness of imp.lc.jented model 
curricula," It is assumed that the sites will try to replicate 
the models as closely as possible. The study ^vas not 
organized to encourage sites to pick out some parts of the 
models and discard the rest, 'nor was it intended that they 
work with the models only intermittently. It is also assumed" 
in testing the effectiveness of- the specified models that 

) 

other factors which may affect the outcomes are controlled. 

Xf there is a large amount of unsystematic input 

to the classes , the purpose of the study will be defeated. 

We v'ill-not be able to draw conclusions about the causes 

of differential effectiveness, A related, but slightly 

different issue is that of, comparison classes,* In Planned 

Variation, many of the comparison classes are drawn from 

the same sites as the experimental classes, vlf these 

comparisons are to be useful, it is important that jthe model 

*- , * • 

not be implemented , in them. Therefore, it is important that 
the two groups of classes not be given the same treatment. 
When we speak of administrative policie*s which do not respect 
.Resign, we are referring to such actions as making demands 

/ >■ ■ ••■ ■ ■ 

on staff time whi^ch take concentration away from implementation^ 
not providing necessary ref>ources to the classes, makir,g 
'non-model input into the classr>oom programs, or cont aii[iinating 
comparison groups. ' * 
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Exampies of directors carrying out poli.cies- which 

* 4. 

interfere with implementation, unfortunately are numerouSi, 

Among them are statements from both OCD con.-^iultants ' and 

sponsors. One sponsor states, for example, that 

The Resource Teacher , . . is interested [ 

in implementing, the [model]. However 

her additional cresponsibi li ties assigned 

by the Director, tend to limit her timfe . ^ 

in thh [model] classes and mitigates \ 

against her effectiveness. She is on^ an' \ - ' 

overload servicing 13 groups, ' /\ 

A similar example of demands -on staff members which* interfe:ij*e 

with implementation comes from a consultant to another site 

who also reports that the Program Advisor 

*^ , ' 

is excellent, but her^ load is^much 
greater than it should be / since 

it includes PV, the comrritment to the ^ 
1969 [mo'del ] , trairiees [who are not the ^ 
same as the 70-71 trainees] and the \ ^ 

total program. As I understand it, the 
total program includes about 50 classes. ^ « 

The important fc^ct^to realiyze is that the pepple r.e ferret 

to in both examples, have pjtrimary responsibility at their 

sites for seeing that the models are implemented. The" 

administrator's decision to make I^Lanned ^ Variation ^ly 

part of the job responsibility.- may seriously hampei: 

implementation. ' 

If ■ , 

Another lype^ of ^ adminis tractive . policy '^hiijh affects 

implementatioji involves demands whiclir tc^ke the staff's 

attention away frdiS^- the modeQ.,- , -Qne consXiltcint', after ^ 

reporting bn a nupibor of dsi-mands that were' be*ing made' of 



the teachers, concludifi's that all the non-model problems 

temporarily detract te^achers from spending 
mental and physical time doing a better 
^ job in the ^classroom. 

Or in another center, a consultant comments about what 

took place in a long^ conference she had with the Program 

Advisor about the model and the practicing staff. One 

of her 'statements is that , ' 

We also expressed concern about the ' 
pressure of. carder development that 
seems to minimize the importance of 
the classroom. . . 

Like the examples above, these support the belief that 

administrative policy can interfere with model implementation 

Another aspect of the requiremients of the experiment^ and 

thus, of implementation which may, be hindered^ by 

administrative actions' is input into the classroom by^ people 

other than the sponsor's staff. One consultant V^eports that 

The Sta^e RTO has ^een asked to come 
in to do some inservice training. ^This 
is a non-model person ., . .She hasn't 
been .asked to adapt anything she will . 
say to the model . . . 

A final problem is that it is ^possible that comparison 

classes may be contaminated because they ar^ included in 

model training. One consultant states, for exaniple, that 

the director cannot understand why -all children cannot be 

included under Plann.'d .Variation. 
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Thus, it seems clear that although we do, not have 

systematic dat^yf(jr all sites, we /can conclude that it ^is 

very probab'le for administrative /policy tc affect model 

implementation. Actions which hxnQer implementation can be 

/ 

interpreted as lack 'of support./ This is not to say that the 
directors consciously interfere with the model. We propose 
that the notion of priori ties /determining support, which 



was presented earlier, is the crucial one here. Unless 
Planned"^ Variation has high priority, the director vill base 
her decisions on other competing interests, and the ' resulting 

policies may interfere -with implementation. In part, this 

I 

may be explained as a lack of understanding of the experiment; 

a director !nay not recognize** that his-'actions interfere 

because she does' not understand the requirements of an 

experiment. The last' example given jDove . i llus trate^ this 

''point v;ell : ^ the director did not seem to have any notion 

of why it was important to include X>n]y cer't.ain*' groups of 

children in Planned Variation. This lack of understanding — 

and its effects — is ^apparent in other ^sites too. One 

sponsor, for example, states that the 

'Administrator lacked clari fi cation of' 

her role, and the meaning of . . . 

sponsorship, -'i last fall a second 

research design was introc^uced into 

[model] class rooiTiS . 

No unders ta'na^^io of sponsorship . 
difficult to ke^^p clear which classes 
are being 'studied , who is being trained^ 
by, . . . [ the^ sponsor ] staff. 



-171- 



Lack of understanding, however, is not the only 
explanation for^ giving other concerns priority over - . 

maintaining the research design. Another explanation can 
be viewed within the framework of the tension between a 
research program and a service program. We propose, first, 
that the demands of a service program axe often in conflict 
with those of a research design. While an experiment calls 
for controlled\inputs and differential treatments, for 
example, a service organization calls for bringing in all 
the resources possible and providing service to as many 
people as possible. Moreover, as the examples above suggest, 
some Head Start Centers havfe goals other than education for 
the children to be met. Second, we propose that Head Start.: 
is considered as a service program, not as an experiment^ 
by those who run it. This is particular''^ true since the 
sites chosen for Planned Variation were operating centers 
before and during 'the study, and will continue after the 
study. Thus, ^ we conclude that the directors in Planned 
Variation sites are primarily service oriented^ and as such, 
may understandably place the Planned Variation research design 
low among their priorities. Such a placement may be inter- 
^reted as lack of support for the model because it may re- 
) suit in administrative actions which inter fore 'v;ith im^jle- s 
mentation. - ■ ■ * • 

In this discussion of administrative support for the 
model we have asserted that administrators have opinions 
about the models and Planned Variation vhich may influence 
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implementation. The influence takes at .least two former 
the atmosphere a director creates for the staff wor)$ing with 

r 

the model and administrative action. We pi^opose/'that the 
second may ber more complica^d because it revolves not 
simply on liking the model but on the interests competing 
with Planned Variation in the administrator ' s priori t<ies . 
If the researc:h design has low priority, action- may be 



taken on the basis of other interests ai^d, as a' result, may 

{ 

interfere with implementation. We cannot^ draw any 
conclusions about the validity of these hyiDOtheees because 

• ■ ■ \ 

we have nc data to test them. .It does appear, however, that 
th^se variN^ have a great deal of potential importance 
^d there :^re, merit further study. 

Summaxy^of Operational Context. Issues ; 

In this section we gave only limited attention ,to 
relevant data. For tt^ two primary sources^ of daii, th^site 
assessments completed by the sponsors and the OCD/ cons; ult ants , 
thej*e appear- to be differences between the sites on a great 
many variable^. ^ The differences, however, cann'ot ..be teste^^ 
statistically. Moreover, there are no significant diffcjrefices 
among models, and the correlations between the ratings 
from the two sources are low. .The data, therefore, can only be 
used tc» s^Aiggest interesting differences among sites; strong^ 
inferences cannot be made from specifiVc variables. 



I 
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In contrast to the data 'presentation , we discussed at 

l^ng^th the ' areas of the operational context which deserve 

further ^bli^. Adminis tratiVa support for the model, 

particularly as it is reflectecj in a director's policies 

and priorit ies^ seems especially impor t ant. We suggested . 

that knowTedge of these additional ^reas, together with 

the context variables for which there are already data, may 

( 

help to explain, variation in levels of implementation. 



SUMiMARY AND CONCLUSIONS ' 

The organization of this chapter is based on the assump- \^ 
tion that implementation' is an • interactive process depending 
not only on sponsor input, but also on staff input and reaction, 
and tine context in which implementation takes. place. Within 
each .of the three major sections, we first presented relevant 
data. In ex.-:uining the^'results of the analyses, it must.be 
, remembered that the findings are suggestive rather than ' ^ 
conclusive . • For all data, there are limitations of using 
sites as f i xed factors , multiple significance tests m non- 
independent measures, and low reliability (as evidenced by 
low correlations betweer^ varia)Dles tapping the same* dimtension 
and ^by lack . of agreement among sources in reports of the same 
variables). In addition, for the training variables there 
»is also the problem ,of lack of specificity; it is no^ 
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clear whethe? the teachers are reporting on only model 
training or on all training. Th^se limitations lead, to 
di f f iculties . in interpreting findings and may result ir^an 
exaggeration, of actual differences. 

From the available data, we find variations among sites 
within models in .all areas and among models in all areas e-xcept 
those measured by the site assessments. 

1. From the variations in sponsor input data, we can 
conclude, ^irst, that sponsors sfpparently differ 

. from one another in the input's thev make, and that' 

at least some sponsors do not- treat all of their 
* 

sites in the same way. Second, it appec^rs ' that 
people other than thq soonsord and their reoresenta- 
"^^^ tives are giving traiViing in the sites. Third, 

there are fairly strongv^ PV-iSIPV differences in pre~ 
service txaining, but not in in-service training 
»or in support ^and fee^dback*. This * finding probably 
reflects a combination of treatment contamination 
an(^ vague^' ins truments . 

2. From the variations in staff background characteris- 
tics, v;e conclude that feach sponsor must work with 

a variety of teachers and that the teachers with whom 
one sponsor v/orks are different from those with 
v/hom' another sponsor v;orks. The relative lack of 
PV-NPV effects suggests that, on most dimensions, 
the di f fercnccs among Planned Variation teachers are 
the same as those amona non-Planned Variation teachers 



The finding of sic^hi f rcjant differences among models 
on teacher characteristics has implications' for the 
fie .on of the study. One requireijient of a ^gocjd 




'w.^porimental design^ is that the situ]it>^ns da v/hich 

li.o treatments are^to be tested do not differ 

/ 

^ s ^ematical l-y r If this assumption is not met, as 

^ it 'is not in this study, \hen treatment effects are 

ironfou'^ded with the other factors wh^-ch^iffer 

systematically among modeXs. \ ' 

■ • 

4. The findings ,for the operational'" context variables 

are limited. From -th^ primary source .of dat^,^ the 

sponsor and consultant site assessments, there are 

no model effects, and site effects cannot be tested. 

We can only . conclude , then, that ther« api^fears to 

be some variation among sites within models on 

. s 

moi. - of the contc^xt variab.les . A factor analysis 

*. . 

whxch ties together some of the site assessment 
variables is presented in Apperjd:.ix E. 



All these , findings support tlie assertion that implementa 

tion is a 'complex process^* The variation among sites and 

models ailso supports the proposition, that these variables are 
I 

useful in explaining variation in levels of implementation. 
In the second part of each of the three sections , then, 
we tried to specify how the variables might be related to 
implementation and to sugacst additional areas which nhou.ld 
be considered in a more thorough study of implementation. 



P'or sponsor input > we proprose that longer and more 
•frequent trarning from the spTonsor may be ^related^^N* - 
to higher levels of implementation hut that Qclter 
dimensions a^re equally^ or perhaps more, important. 
The typ^s ^pf training 5given ^and the extent of 
pers^onal (support and feedback from th'e spons^ may _ , 
also influenc-e implementatioh . Moreover, ' the ^ • 
present data suggest that we need ^ta further explore 
training by people othei: than the SDonso'r: such 
/additional training may- either facilitate or inter- 
fere with model implementation ^-^dependina on who 
<^ves Uie training and'^hether it's consistent 
w^th the model. ^ Finally, anecdotal evidence 
suggests that sponsors may betom^ involved in 
activities in the sites which rajige beyond model 
prescriptions. Since it is possible that the§e 
activities may influence implementation , perhaps 

through the> degree of commitment felt by the teachers 
\ ' . ; 

the- area should be explored. {■ 

Under staff input and reaction, we propose.- that not 

* ^ • . ♦ ■ * 

all teachers have the same inclination or al^lity 
to Implement a model. If staff characteristics 
determine the difficulty of and motivation for imple- 
mentation, they nii.ght help t:p explain the variation, 
in levels of implementation . We suggest- that reaction 
to training, age , experience, level of ed\«cation, and 
certification may be determinants; of levqls' of 
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implementation. There are no data, hcwevef , for 
'the other variables which may b,e crucial to the 
relation' betv^een staff characteristics and imple- 
mentation: liking fpr the model as an approach to 
teaching, the extent to which model requirements 
are understood, and* previous methbds of instruc- 
tion. Tf a teacher likes and- undeorstands the model, 
and if it is consistent with her approach tc^*t?aacJaitf«^^ 
we. predict that she wi-rl have a higher level of 
implementation than sho would othenvise. 



3. We suggest three areas of variables within the 
operational context of the sta^^^category . Under- 
the heading of site administration, we propose that 
unless a sil^e, is operating e-f f iciently , the enargies 
of the .staff will be diverted to adininis trati ve 

i , 

problems, away from implementation. Site adminis- 
tration^is reflected in such variables as staff 
stability, staff friction, administrative competence, ^ 
and funding security. ^ ^ 

4. TVnother important, but neglected area,' is administrative 
support for the model. On the basis of anecdotal 
evidence, we propolse that the administrator's opinion 

, *of the, model anV of the Planned Variation experiment 
will influence implementation through the atmosphere 
she creates and throudh ^lor actions and policies. 
We also assume that the latter'^means of influence 
v;ilJL be more complicated because it is dependent on 

. . • • / ' ' 
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the director's priorities for her prograir^ as woll • 
as on' liking the model. Purser, we assert that 
we would expect to find that Planned Variation is 
a low priority concern to many Head Start directors 
because their primary conqeption of their program 
•is as a 'Service project. As a result, we would 
' \ also- expect that some of the director's actions 

would interfere with model implementation. Finally, 
we propose that site ch^aracteristics , such as ^ize, 
adequacy of facilities, the delegate agency, ar?d 
the values* and orientation of the r^ite may be 
important to implementation but the nature of the 
relationships are not always as clear as in other 
areas. ' ^. • 

The final conclusion to tijis chapter is. that we have 
raised many issues but resolved few or them. The. finding 
of model and site differences on the.majority of variabte^ 
supports the argument that implementation is a complex 
process and that some of these variables may be useful in 
explaining the variation is levels of implementation. There 
are, however, many dimensions of the implementation process 
for which there are no data. ' 4. 
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Chapter 4 

« 

PREDICTING LEVELS OF IMPLEMENTATI ON 

*' ' ' • 

In Chapter 2, we presented the sponsors* ratings of 

teachers as a measure of level of implementation. We 
concluded' that there is a great deal of variation among 
classes and sites within7iTiodels in thp extent to which 
models are implemented. In Chapter 3, we explored a 
number of varia]>les which might influence levels of 
implementation. From the data available for these 
variables, we found large variatipns botii among and withm 
models, and predicted that the variables nvidfht help ex- \ ^ 

.plain the variations in levels of implementation. > V 

♦ *"^In this chapter, then, we will use multiple regres-*- 
sion analyses relate the variables discussed in Cbapt,er . 
3 to the sponsor ratings presented in Chapter 2. 

Multiple regression, analysis is a technique for 
estimating a functional relationship between one dl-ppendent 

variable and several . independent variables. The- result-^ 

\ 

ing equation can be used to predict values or the dependent 
variable for given values* of the independent variables . / 
The equation for'^ linear 'regression can be expressed as: 



Y 



b^ + biXi^'-f- -b^ x^ + ' ' ' b X y 
o ll 2 2 . n n 



where Y stknds foif the score on the dependent variable 
(sponsor ratings) v:hich will be predicted from the 
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knowledge of ,a set of independent variables; x^, y.^, . 
Xj^ stand for the given scores on the independent variables ; 
^1' 'r^'^^n coefficients of the independent 

variables which give the best linear prediction of the 
dept?rrd^t variable; and b^ is the regress ion^onstant . 1 
In ^ur case, the sponsor ratings of teachers will be 
used ^as the dependent variable,^ and the va^riables > intro- 
duced in Ch'apter 3 will be used as the independent v-^riables, 
From the regression analysis we can also obtai; the square 
of the mult^iple correlation coefficient (R^) which is the 
amount of variation in the dependent variable explaiined 
by the independent variables. is an important ipdica- 

tion of success in predicting Y; one piTimary goal qf the 
regression analysis is to maximize R . 

In attemp^ting to explain the total variation' among 
levels of implementation, we can separate tlie variation 
among models^ among ^ si tes witMn models and among teachers 
witkin sites. Considering firsV the differences among 



The size of a b coefficient depends on the scale in which 
the. variable is measured. As a result, comparisons of the 
size of the coefficients cannot be made. To overcome this 
problem, we can use 's tandardi zed coefficients, b^ . For a 
m.ore >thorough discifssion of* regression analyses', see Draper 
and Smith, Applie d Riagression Analysis (Nev; York; Wiley, 1966) 

In Chapter 2, the ratings f or ■ both February and 'May were 
examined. Since the May ratjL^ngs were made any time from 
May to the following October, i^^ks decided to use only 
the 'February ratings' in these analyses because they are 
more representative of the overall level of implementation 
in the site during* the year and are probably more valid. 
The analysis of variance of sponsors' ratings f or^ Februar?^ 
only can be found in Appendix C, Table 5, 
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models / we would not expect a knowledge of model^^ ex- . 

plain juuch o.f the variation in levels of implementation 

because in the analysis of variar^c-e of sponsor ratings, 

J"^ models explained only 3.1% of the total variance. As a 

check, however, we entered eie\?'en of the t\^^elve ^PV •modols 

as dummy variables 't6 explain variation in the total ' " ■ 

'syst'^m* Table 27 shows the results of this analysis: 

. ^ 

models explain only '3.1% of the variation in sponsors' 
ratings' of teachers . The F-test for this value .is not . 
significant (p = . 275). In fact., only one model., REC , * 
has a significant b coefficient (where' "significance" can 
be described in terms bf the extent to which a variable 
significantly adds^'to the equation predicting the dependent 
variable)."^ Ive can conclude, then, that model effects 
do not help to explain vajriation in levels of implementa- 
tion. Therefore, we will ignore them in the rest of our 
analysis,.. 



"-"The use of dummy variables is a technique for intro- 
ducing variables v;hich are not conventionally scaled into 
a regression equation. We define one dummy variable for 
each m:>del: all teachers- who work with a given model re- 
ceive a 1 on the corresponding dumjny variable, and all 
other teachers receive a 0, Only eleven models are entered 
as dummy variables because the twelfth is redundant in 
the sense that it adds nothing to the amount of explained 
variation, Mol-'fover, any set of ele n models yields the 
same result . . T 

^Inspection of the model means on sponsors' February 
ratings (Table 5, Chapter 2), shows that RKC has the 
highest mean, but the- model only has one site; there 
are other individual sites v/ith equally high means. 
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TABLE 2 7 



Regression Analygis" - ' y 

Model Dif ferenesGs j^n Explaini^nq Total y-dr.iatio s?' 

in Lov e ls^oi I mp 1g men tat i on' ' ^ ! 
- .11 . ■■ ; — ■ y — — 



Constant 



R 

R 
SD 



res 



= 0^.081 
p. 2,85' 
= 1.724 



Variable Name 


■ • b 






b* 


T-Test" 




Siqni f icanc 


Far West 


-0.1865 ' 


■ '■ 0 


. 530 


~ — 

-0.0 38 






>.5"00 ■ . 


Arizona 


0.'012 8. 


B 


.55 5 


0.002' , 


'{ ■■'0.02- 


1513- 


' 50O ' • 

t 


Bdnk St. 


-0.4535 


0 


.'547 


-0.088 


-0. 83 


15 3 

« 


- .409' 


Oregon 


0.59 71 
0.0128 


' 0 


. 630 


0,.091 


0.95 


153 
I 


.345 


Kansas 


' .0 


,676. , 


■,0 .002 


\ 0'. 02 


.- 15.3- 


» ->:-500.'- 


High Scope 


-d. 2244 


• 0 


. 5'59 


• ' -0.042' 


' '-0.4O 


1$3' 


■ > ..500 . ' 


Florida 


0.0 85-5 


0 


.030 


0.013 ' , 


"0.14. ' 


, 153 


>..500 


EDC 


0;6l28 


0 


.690 •' 


• \0.00 2 ,■ 


. 0.02 


15 3 


' " >'. 5.00 


Pittsburgh 


.134 2 


. 0 


.916 


-.0.012 


-0.15' . 


,15 3 


>.50Q 


REC 


, 2. 422''4 • 


0 


'. 8t9 


'0/2 4 di 


2y. 79** 


• 15 3 


, .'006 


NYU 


0.0128 


" 0 


.746 ' 


0.002 ' 


0.02 - 


■15 3 


■ -^-...'500 


Regression 


' 5.'l0 2- ' 















F = 1:^3 v/ith 11 and 153 degrees of 
f re e dom (-P = . 275) ^ ■ \^ * , 



Notes : b = beta cc;efficicn.t .' , *' 

■ b*=?= s t andardizt^d coof f ipieiit. 
SE, = standard 'error of the coefficient 

R = multiple correlation 

multiple corrolatio]^. squared. 

SD = standard deviation of residuals - 
res 
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^ i • ■ * 

\ 

•. Variation Among Si toy : * | 

It is apparent, then, th^»t the variation in l^-^vels 
of implementation lies among sites and among teachers 
within sites. We are interesto^d, first, in difforc*icos 
among sites ^ with sites considered both- as i ndopo.-idon t 
and as dependent variables , At one • level of' anal ysis , 
we can use site differences as i ndeoendent . variables to 
explain the total variation in sponsors' ratino]s., At 
anot/.er level, v;e can use sites as dependent vari:aij]os 
and try to explain the differences among them. 

In order to deal only with vnriatio/i amono sites, 
as opposed to variation in the total system, we will 
consider variables on the "site level''. Some variables,, 
such as those from 'the site assessment, a^re. already at 
the site level because there is only one observ<:fti c n 
per site. For other variables, including the sponsors* 
ratings of teachers, however, the original data has 
one observation per class (and consequently, there is 
variation within sites). For these variables, including, 
the dependent measure, we vv'ill use site means in estimat-. 
ing si te effects , 

y 

Because it is impossible to enter all possible 
variables in a regression equation, we- selected interesting 
variables from among those which correl-ato . ino'st 'higlily 
with the site mean of the sponsors' ratings pf^the te.acUers ' 
performances . Table 29> shows the highest correlations , ' 

■ ■ '■ i 



t 
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TABLE 2 8 ' ' 

'■' if 

Site Level Correlations 'With Sponsor Ratings of Teachers 



Training: 



SI#4b Number days in- 

SItt4b NurribG,r days in~ 

SI#4a Nuniber days ,prG 

SI#4a Number days pre 

TQ#11 Wliether teachei: 

SI#4b ^ Numljer hours in 

TQ#9 ' . Whether teacher 

SI#4b Number hours in 

TOJ'6 . ^^ether teacher 

TQ#6 _ Whether sponsor 



service traihing for aides ..58 

service trsini*ng for,, teachers " . 51 

-service training for aides .43 

- s e r vi ce t r ai pi n g f o r te ach q rs .44 

requested, help ■ ' ■ ,38 

-servi.ce training for aides .37 

had., group, .discussion in in-soxvice . 35. 

-se rvicc training for teache rs . 34 

had grou^ discussion in ore-service . 23 

gave- sofpe pre-service .29 



AQ#26 
TQH2 
TQ#38 • 

Aono 

TQ#40 
TQ#36 • 



Staff Input : 

Helpfuln^^iss of aide ' s training 
Teacher's years in Head Start 
Teacher ' s race 

Aide ' s satis faction wi th equipment 
Teacher had K-2 practice teaching 
Teacher ' s age " 



54 
48 
45 
44 
37 
34 



J Context: 

( 

SI#3.d Intra-staff friction^ / -.80 

S.I#3.n Rapport between administration ^and , staff . .74 

SI#3.o Ratpport between staff and children' . .62 

Sl#3.p Rapport batv;een' sponsor staff and local staff ^ .46 

'■"SI#3.k Adequacy of physical plant indoors 40 

CR#15 Rapport between admins trator. and staff .40 

CR#4 Intra-staff friction. - -.39 

•SI#3.1 Availability, of sponsor guidance ^ .39 

^ .SI#3.j .Support, of PAC for the model .39 

++ ' Whether site housed ar d adminis tei'cd by ptiblic-v^ ^..57 

school ■ ^ ■ • >^ ' 

CR#6- Puctuality-'of teachers ■ ,^ ^ . . " ^ ■ 36 

,'.SI#3.h-' Suppprt'XQf local Head Start oersonnel for., the niodGl - r35^ 

TQ#24 ^Teacher sa ti':i^.f aCti(?)n v;i in, v;Oi;kiny- condi'tiairs^ \' . ' J-'-.'SS^ 

*\ CR#7 "^Rt^gula^'. attendance • of children • ^ . . .'^32, 

Sijf3-6i Support, ^of community, for tnj model.' . — - . ^^,2 

SI#3.k-- Adequacy of-.phy^ical plant outdoors ^ ^ * . 31 

+H- V ^ Whether site^hou^sed by ^"'^ ,"-30 
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??.ar3on prcduct-T.om: 



correlations 
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pj/hich are divided into three groups corresponding to the 
three domains in Chapter 3 (with the cxcpption^ that sponsor 
inpu? will be defined more- broadly as training,). In 
orSer to explore the largest nunber variables-, wo 
first ran a separate equation for each domain.^ We 
then took . the variables w:. th the largest coefficients 
from each equation and combined them into one equation. 
In each case , .the regress j^OT^,^,^^^^^ s tepwise , " Under 
this. method the varj^ables alr^ enter*^-- one at a tine, 
in order of maximum improvement to^ the regression euua- 
tion; the variables wKich 'add most to the multiple 
correlation squared are, pntered first. This j, process 
can be stopped when the. last^v^i^ble adds less than 1?. 
to the multiple, correlation, »or when the standard devia- 
tion of the residuals ^s at- a m.ininam. Tl^ie final equa- 
tion for predicting site-niean levels of implementation 
shows a Striking finding (see Table '2/^). We can explain 

of the varial on a?Jiong site s iii spo nsors rati^'gs 
(r2 = 0.9 77") with th ■ ki-owledge of t hVee variabl es^' th e 
a mount of friction w it hin the site^ th e, rapp ort bet ween 
■ the administration and the staff , and"; the .ad^qy'acv of the 
physical plant inside--all as judged by' the ^>fyon-sor-. 



(The ratings on the sane dimensions by the OCD consultants 



^When the numbdr of predictor variables is large in 
relation to the nycnber of observations in tho study, the 
estimated true validity of the regres-sion equation, is • 
very lov;. The three equations are .;?*iOv;n i.n Appen'dix 
C, Tables 6 , 7 and 3. 
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f 

dp not enter) . The friction variables has a necrativc 

coefficient, v/hich^ indicates cnat a low level of friction 

is associated v/ith a high level, >of implementation. The 

other two variables have posij^ive coefficients, which • 

indicate that high ra'pport and good facili-^ios are as=:sociatiied 

with high implementation. Clearly, these variables 'togetlior 

present a picture of a pleasant site. Moreover, because \ 

the friction and rapport variables alone explain 90. 6 o^f tlie 

varian^^e, it appears' that harmony is the primary - component 

of a ^"pleasant '^rte". 

Several explanations of this' finding are plausible* 

It is possible that sponsors are' simply equating ple.asant 

sites with high levels of implementation: ^ site differences, 

» * 

may reflect only di^tf'erences in pleasantness, and may be 
unrelated to actual differences in per form.ance with ' the model. 
This interpretation is supported bv the faot^ that consultants' 
judgments of the same three variables (friction, rapport, and 
adequacy of facilities) are not re'lated-^to levels of , imple- 
mentation. It is possible, however, that pleasant ^ites 
^actually do have* higher levels^ of implementation than do 
unpleasant . suites . This may nTean, on the one h^nd, that high 
Devels of iiaple^ientation occur in sites v;hich have good 
facilities and good relations among the staff before the intro 
duclsion of the model; a pla. i\nt ^ite may be a prerequisite 
for implementation. On the other hartd^, it may be that the 
model creates the pleasant pnvironment: the staff in ^ site 
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4 



may be so enthused by the model, that they work well 
together in order. to implement it . ^ Although the available 
data do not allov; us 'to rc5olve tlie difforencos in interpre- 
tation about this relationship, the •importance of this 

^ 6 
finding should be j^ecognized. 

/ 

Variation Withi.n Sites : 

Explaining almost 'al 1 of the site -to- site vari ation 
and knowing there is ''almost no model-to-m^del v.ariatio;i 
leaves, us on Iv with classroom within site variabion to v - 
explain, ^ To deal with withan-site variation, we will 0 
use a class iTevel anaJ^sis. At this level, however, we ^ 
are dealing v^ith all th'b var£^ation in the system, includj:ng 



Three other variables--the aiders judgment of the help-' 
fulness of training the sponsor*s report of the nuKiber 
of days of ■ iH-5ejvice training given to the teachers, and 
the sponsor's judgment of the rapport between the- Planned 
Variation staff and the children-- were in the analysis but 
did not elTber tke equation because the large partial cc^r- 
relations mean that the correlatiqn matpix is not positive- 
definite. ^-"^ 
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site-to-site variation as well as within-sito. One 
stratfegy for renoving site effects in order to^deal only 
with variation within^'si tcs would .be to enter all sites 
as durnjny variables, as, wo did v/ith modol-s, • This, however, 
would use 'up a largc^ number of, ogrecs of f rcfcdor'i 
(number of sites - 1) -and the subsequent estimates would 
be less precise than if few^er degrees of freedom had oeen 
used. An alternative strategy; and the one which we 
followed, is to use the three variables which exp]ain 
98%^ of the variation between sites (staff friction, staff 
rapporr,,^ and adequacy of the physical, plant) as proxies . 
for Giteo- Table 30 shows that these variables account 
for 2^,>j oL .the total variation in levels of implementa- 
tion. ' ".f .wG force th^m in an equa^tion fi.rst, we .emove 
the ^ I ■. ■ 1 o-site variatibn and are essentially left with 
within : { te variation, / "^^^ 

Tnr- nighos^t class-level ^correlations betvs^een levels 
.iinontatioh and a range o^^^independplit variables 

in the? :.'^:.ee domains are shown in^TabJe 31. These corrcla- 

<»• \ ' 

^ ' . \ 

ticiis obviously lower than the correlations at the- < 

, ■ ■ . \ ■ 

site ] • 'Vel . We f ollowed* generally the "^.ame procedure in 

deve '/"»ping within-site equations as we di^ with the site 

levej . IJovvGvor, bec£i. . j *of the small numbcKof even 

\ 



7 * 

'It sr'->'ild be noted \ ..it: in explaining 29.8% otn total 
variar with si tc y a r "UDles , we ha\/c excoedod th\ amicunt ; 
expect.: ! o:. the basis of the amount ^ of variance du^^'to sites 
in th. ..naly ,s or variance for February sponsor ratliigs 
{Z2'ij) . 'Tills is probably the- result of different data\ 
'Jbasc^: the regrv-.ssioris are based on. daun from all Clas'ses 
for which there . -re sponsor ratings > v;hi2h' is more dat^aX. 
than wore included in the balauced ano':a (in standard dcjGicn) 
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TABLE 31 



Class LgvgI Correlations With 
Sponsor Ratings of Te a ch g r s 



TQ#9 

T0#11 
TQ#6 

TQ#9 

T0'#5 
TQ#6 



Sponsor Input : 

Nuinber types of in-SGrvi ce training given by 

locq)i Head Start office 
Whether teacher requested help ♦ ' 

Whether local Head Stare office gave some pro- 

'servi ce training 
Whether teacher had group discussions' in in- 

servi ce training 
N'umber days pre-service training received 
Whether sponsor gave some pre-service training 



r 

22 

18 
17 

17 

15 
15 



Staff In;:)ut : 

TQ#42 7. •"^:?.-:her ' 3 ye ars experience in 
TQ#3b Teacher's age 
TQ#38 Teacher's ^ace 



Head Sj^art 



26 
22 
21 



Cont ext : 

TQ#39 Degree of parent involvement 
TQ#24 Teacher's satisfaction with v;orking conditions. 



15 

13 



Note : These are l^^arson product-moment- correlations . They 
include all scores, not jufet relations v/ithin sites. 
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moderate correlc;tions\\ we only used two equations to begin 
rather than th^r-ee: one equation with training variables, 
and the other v/ith both st^^^ input and context variables. 
Tables. 9 and '10. in Appendix C 'show these equations.- 
It i^ apparent that' we cannot explain much, of the ^^^i thin- 
site variati on in sponsor ratinQS wi th ei ther of these 
equations. Knowledge o^f ttaining variables adds 8.1^ 
to the e>^lained variance above site dif C,erGnces , and 
knowledge of three staff input and context variables 
accounts for 1.4% of the variance after site-to-site 
differences have been removed . v 

It is also apparent that, the .adequacy of facilities 
variable is not adding- significantly to the equation. 
Since the goal of a regression analysis is to explain 

lost variation with the fewest vari^ables, this vari- 
atXle ya^s dropped from'*the analysis. In the site level 
equations, removal of this variable reduc^.s. the percentage 
of explained variance from 97.7% to 91.3%, but inlthe 
class-level equations, it oiily redi5ces . tHte percentage of ' 
tptal explained variance from 29.8% to 29.6%. Thu-. the 
two remaining variables, staff friction and staff rapport:",''' 
can still be considered as proxies for between-site . effects . 

The final equation was formed by first enter i no the 
two site proxy variables, and then allowing the best 
predictors from the previous equations to enter in order 
of importance. Table 22, which contains the results of 
this regression analysis, shov;s that even v;ith the strongc\^i 




variables combined, we can •o;Mplain only 39% of tho* total 
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va-.L.iation in sponsors' ratings of levels of implemontation . 

> 

More impo rt antly / we can only explai i 9.45 of the variance 
with wi thin-site variables / after site effects have i)Gen - 
removed . The five variables v/hich' contribute to the^9.4 6 
^ do hdve significant coef f icieats , .however, and deserve to 
be considered individually. All five variables come * 
from the Teacher Questionnaire; four relate to pre-, 
service and in-service training wi... is a teacher ^ 

character is tic . 

We -win cor: id'.- '^ne teacher characteri sbrc^ number 
of years in Head Start, first/ even though it is the 
last variable to enter, because it-^ is the easiest to 
interpret. The coefficient for this variable is negative 
indicating that a teacher witli more experience teaching^*" 
in Head Start tends to implement the mod(^l to a lesser, 
extent than <a teacher, with fewer years of experiehce. 
It may be that a welL-established tea^cher ha's l. comfortable 
pattern of instruction, whir^lvsjie does not want to give up 
in ord: to fully implement a^mcdel.^ 

./ i'he relationships of training variables to sponsors* 

. I 

ratings of implementation are more difficult to^deal with 
because it is not. clear whether the v;ithin-site variation 
on ihese measures results from real differences in e.xpe\-i- 
encG or whether they reflect differences^ in memory or 
perception of the same experience. ^If we assume reaj. 
differences, v;e can conclude from the two variables with 
positive coefficients tHat the teachers who received some 
♦ 
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pre-service training from sponsors, and'^who participated 
in group discussions dujesLng* in-service training have 
higher levels of implementation .Thi/ese . findings seem 
reasonable ■ It is ^ogical-~and.; in' a s^:?nse, reaS^suring-- 
that sponsor training has some impact on leve Is of ample-- 
mentation. Perhaps-, the second v^.riable is importe.nt ' . 
because teachers need to 'talk things over in order to ' 
gain mastery of the model, -From the two training 
variables with negative coefficients (still £issumi.aq 
real dif f erences ^'n ^Bxperience) , our conclusions ar^*' 
less clear. The regression analys^^^s indicates that teach- 
ers who receive trainirlg , ' c-otfr in-s^erv^ice and pre-^servico", 
from the .local Head Start office tend :,o^have lower 
levels of iinpleirientation than^eachers who did not re-- 
ceive training from this source. It' may be that local 
training interferes with implementation in some way, 

4 '■'■■-/ 

A • o • 

either by directly' contradicting the'itodel', or by adding 

inputs which oveirload .and confuse the teachers. If the 

.seconcl interpretation of within-site variation is correct, 

that the variation comes from different perceptions rat^e;t.-" 

than different experiences, the conclusions* are slightly 

difiereat. We might speculate that the teacher who 

imp^ments the model well is on^'to whom, .sponflor training 

is import^int. As a result, she emohasizes it mo2;o in 

her ireciiil of\w^^ hap^^^ened than dp other teacher's: . 

Converseljy^ , a tether v/ho does not implement the model *??iay 
. \" ' * 

be the on.e 'who has focused oi^ local traini-^^j^ and, -therefore 



reipembers it more. The^/Causal .sequencing of tho^^^*. 
Jreia^iiionships is not//cle,ar. Recp^rdless of interpretati 
• hov/evej;; -the findi^rig that , twining from the local, Hcad^' 
tart of f ice .is/ negatively s^elated to iir.plomenta\ion is . 
import an-t ^^^''^ • ' , \ ■ , ^ ♦ ' 

■ ■ ^- ■ ^ .■ ., 

SUMMARY • / * - ■ . ^; ' ^ 



Using regression., analyses to explore the relation bctw9on 
a measure* of the extent of model implenientiation and a va^fifety 
of independent variables, we H^ve made a number of ' findings .\ / 

1. We can exuJ.pin ofily 39% of't:hG ^k?:iation in ..\ieve Is ' 
. of implementation ..with the data av^i lab^e '^'to *us. 

in this study. ^ . - . - . ^ 

2. Model-to-giodej. difference^ exp.lain an insignificant / 
amount of the v>&riatiori : ' , ' ^ • . , 

3. The wi thin-site -variati'^n^ is '^th(^ lar^st ^artv.(!)f the* 
. total, variance , but we cannot °e^pla\n it well :| alter- 



^ cOiTv.rollirfg for site-to-site di. f fe^'enfcesv "^Ibss than . ^ 
10% of -additional variance ca;) betaccountfed for. The ' 

J .five variables v;hich contribute to 'this 10% are,, in 

' \ . ■ . . . - . . . 

* order of importance, the^ number of types^of in-service ; 
tytaini^lg^ Ojiven by- the loc:^x .Hc-^ad Start- officey v/bether 
there ,Ue£^e group di/4J[^^sions durin(^ in-service ^rainiao , 
y^^vjhetYxQr the, sponsor gave some pre-^servi ce training\|; j 
v/hether thQ local Head Start office aave some pre-^-\ 
service traininq, ' and. the teacher's- years of '6x901^1 eVncc- 

•■■ ■ V ■ ' ■ ■ ^; ■ . \ r 

in Head Sta'rt, All of these variables j3dn]Q from the \y 
Teajjher Questionnni.ro and\there f oiire ^ have withi.n~p.i tg 



variation. . * ^ ' . ^. . 

, - k' • * 

The* njost impressive findlno^s, tliat 97.7% of- the 
'site-^tp-^si^e variation in levels of implementation 

can be explained v/ith three variable's, all of-wfiich 

''are sponsor's judgments: intra-staff friction; 

'rapport between the site administration*' anc3 staff, 

and the adoauacy of the physrc^al plan? ijidoo^s.- 
0 ' ' ' • 

Although several alternative interpretatidn of this 
finding are po^ssible it appears <' that it indicates 

■ ■ ■ ■ : ) 

"that sponsors are ec5f'uatin-g pleas^^nt sites with high 



levels of im.plejnentation . 
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Chapter 5 
CLTlfeSROOM OBSERVATION 



This chapter r^ports'how raqdel classrooms differ frpm 
• one anoth^er in practice on several npasures of . adult and ^: 
" *' child behavior. The first s:ection describ^^he' classroom 

^ '■ . .. ' * ^ V \ . 

observation, instrument*", the f if ty~one '-depende-nt variables 
derived from it, and" the statistical^ anal\^sis performed on 
thes'e variables. The .second section relates , the results- of 
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the analysis on tv/enty-two vari^le|^--~thc£e vari,ables which- 
se'emed to reflect genuine differences ^mong models, -In t>^e 
third section, we attempt to summarize each model- ia terms 
of its -^scores on variables which should represent im.portant^ ' 
'aspects of model theory, and in terms of th^e . y^ariabies on 
which each model is significantly different frbm;other models. 
Thu3*, v/je^report the 'd^ta; in two v;ays: fiirst, in a straight- 
forward^ manner by variable, and second , ^ in^' an intS-rp^r-etive . 
manner, by model. , ' ' * 

A /comprete list of the- fifty-one" variables is included 

F ■ » ■ ^ 

in Appendix F , and tables of meane and standard deviations 
for all eleven models on all the variablGs' a5;e in Appc^ndix 
G, Although detai.led' desctiptions of model theories are 
not included in this chapt4?r, the reader can refer to "Design, 
Data Collection 'Activities and the Curricu^m Approaches" 
for more infarraatich* about each sponsor, ' • 
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CLASSROOM OBSERVATION INSTRUyjMT 

The Classroom Observation Ins wrument > ,a s,tanda,i'dized 
* ** . * * 

instrument designed by SRI 'to assess child and adult behavior 
and interactions , tyipis of activ>i>,y, ^ general •atmosphere , and 
physical equipment in'^a classroooiK Unfortunately, the ; 
yfeiTiables u'sed' in thi^ analysis are. .derived from only tv^^o of 
the six parts of the instrument . TJie&« two- parts, ,the 
Classroom Check List and the f lye-minute' observations,, dc- 
scribe typ^s of activity a^nd- interactions . On the Classroom 
Check List, the observer codes all individuals .and their 

activity for one, minute intervals fox^j: timfes an hour. The 

' . \ ■ ■ '■ ' . • . t: ^ 

. five-minute observations, consist 6f coding the interactions 

of ,a selected group of children and adultis iin a specific 

•activity. " The coding,. is ii^ a sen-ten-ce format with four com- 

•pon*ents.:whQ-Tto wifem-^does. what--~and hcwi'it is done. , For - 

example, a teacher (T) to one child ,(C); ask-s a direct ques'- - 

tion (1) firmly (F) . Thus, th^ code sintei;ice would be 

T-C^l-F. Or,' a small 'group of children (S) among themselves 

•(S) "^lay (5) happily (H) S-S-5-H . AVi. observer is expected 

to complete Op to- sixty frames (one frame includes one "code 

sentence") per five-minute observation or one frame every . 

five seconds. ^ * \ ' 



l.TJhe full Instrument and' inistructions for its use are de- 
scribed in SRI, Im p ler>\^. Tit a tion of Planned Variat ion; in 
•Head Start (1969-1970), Acpondi.: B. . » 
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Observer^were trained by SRI and assigned t6. visit, 
each Planned variation "si^e once in the fall -of 1970 and 
once in the spriivg of 1971. -Each observer spenV ^'two mofn- 

. ings in up to four Planned Vari&tions classrooms ahd up to 
four' $:omparison or non-PV- clas^robms , While the same ^ ^ ^- 
pe-rso'n "observed both^PV ^hd r^on-PV. classroouis at .a site., in 
about half of the sites, a differo^nt por&Cn observed in the 
fall and spring. *PV classrooms were sel^'c-ted '.fo'r^ bbservatipn 
on the basis of -the teachers' years of experience ^ith the • 
model. In other 'Words/ SRI chose those/ classrooms which- were 
Titejre likely to be well iipplepient^d , b^'sed 'on the hypothesis 

-that' teafc^rs with' more exp;qrience in thc^^odel are better , ' 
abl(= to implemej:)1:> the- speglf icaticns of ^the model. COTipari-* 

. .son ■ classrooms were selecited" on tha basis of th^ li-kelihood 
that* the children would lster .be in Follovz-Through- blassrocms , 
in order to increase the probability .of a large sample in' a' 
foMow-up stii'dy.* • ' , • \ 

CTur a/ialysis th<as Includes a fairly large amount of data 

'':on PV ^na^ C9mparison >cl ass room's . . Although the data do not 

cover all of the aspects of classroom behavior that we might 
I 

want to examine, they do provide information about various 
kinds of a,ctivitie£ and Ijow child'ren' and adults relate to one 
anot^^ier . « • • ✓ ' 

•r 



• \ 



i 
I 

I 

A 
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VARIABLES .'" ' ' ' * ' ' • 

The- f if ty--6ne dependent variables ^ on which our .anallySis 
is based ar^ only a fraction of the entiiTe *group' of variables 
' ttiat could^ be exLtr^cted from the classroom observation data.- 



These fifty-one variables were designed to represent import--* 

ant aspects of Planned ^^-a-ria^ ions models. Each variable 

* • . f ' ' ' . . . ' ' ' 

. should, theoretically, reflect differences among models if 

the models are implemented according to the; ' sponsors V ^spe-' 

^ cif ications. ?>owever, many of the Variabiles are relevant* 

* ' - * 

to TO r a €han one model: for example, six sponsors, emphasise. ^ 

"i'ndispendent child activity" and no -sponsor stresses "large 

group* activity. " (For "further examples, the reader can refer 
• « *' « 

' to' the tables of theoretically important .variables for^ e^ch. , , 
model m the Model Summaries section) . Thus, since several* 
of the models, have more similarities than differences in . 
their^theoriea, "one cannot expec^t, ,that each variable should 
differentiate every modeT. One can expect, however, that 
most of the variables should show differences among ^ome 
models. - , _ ^ 

There arie two furth&r qualifications, to the statement 
that the vari.ables represent aspects of model theories. First,- 
none of " the variables is relevant- to twomiodels: University of 
Flprida stresses training parents to teach in the home rather 



A list' of all fifty-one variables is'inc^uded in , Appendix F f 
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than teaching children 'in the classroom; Enablers have no 
prescribed theory, but gather "enable'** Head Start teachers 
to ^implement an educational) program based on 'the needs and 
desires of the p4rents and^ community . 

Second, because the valriSbles are derived from the 
Classroom Check List and fiVe-minu-te Observations, both of, 
which report readily observable behavior ^nd .require rapid 

* ■ ■ • . ■ ! 

coding, the variables are more relevant 'to those models i 

which specify particular types of activity and interactions, I 

,In other words, we cannot construct variables from the Class-- 1 

• ♦ 

room Ch^ck List or five-minute observations which mfeasure I 
whether a teacher , is responsive to children's needs, "or 
whether a child is ale'rt and interested in the classrgom 
activity. In contrast, we can design variables which rep;re- 
sent "academic activity," "children* asking questions," ."in- 

^^.pendent play." T5:\us, if a model specifies a certain amount 
of acaidem.ic activity or emphasizes independent child activity*, 
we have data on ^hicli to determi;ie whether, in. fapt, these ^ 
types of activity are occurring in ]:heir classrooms*. However, 
if a niodel specifies no^ particular type of activity or curri- 
culum, btit: rather that adults should 'aim to provide a setting 
in which children are encouraged to make their Qwn discoveries 

'we do not have data on which to -judge whether this setting is 
actually present, "tiieref ore , the analys^s^^jprf these data can- 
not be used to ansv;er questions such as, "Which model is most 
successful in implementing its theory?". This analysis does 




answer sUch questions as*, "Which model has more academic 
activity in its classrooms?" 

STRATEGY FOR FINDING MODEL DIFFERENCES 

In order to^find '^"model effects" or, si<5nificant dif- 
ferences among models on our variables, we first examined* . 
'means and standard deviations by model^' and by site^ • 
Then; we ran an exac^ least-squares analy-sis-^ which inclijcied 
all the data and provided F-tpsts for model effects,, site-^ 
within-model effects, PV/nonrPV effects, PV/non-PV by model 
interactions, and PV/non-PV -by site interactiorjs for. all , 

• *♦ 

variables . 

-.*■*' ' 
] .Originally, we planned to us^' PV/non-PV Effects as a 

criteribn for substantial ^odel effects, under the assump- 

tion that if non-PV classroom- scores v/ere like PV class- 

. room scores *oh some measure, then the pV scores did. not 

indicate model influence^ Hov/ever, several factors persuaded 

us to change that strategy. First, the same person observed 

both PV and non-PV classrooms at^ a sitei and if that observer 



11 models with PV classroomsr fall and spring; ^ ' 

9 models with non-PV classrooms^: fall and spring. 

29 sites with PV clas^rocms^: fall 
26 sites with PV classrooms: spring 
, 20 sites with. non-PV classrooras : ^fall / 
18 sites with non-PV ' classrooms : spring 

Developed by Jeremy D. Fifin of State University ■ of- Nev; York 
t at Buffalo. • 



coded ^differently enouigh -^rom other observers, then the lack" 
of difference between PV and non-PV classrooms could be a 
measure of observer effects. Second/ v;e found, that some- PV 
classlrooms were in the same building with non-PV classrofoms . 
and*'th^t PV teachers ex-changed equipment and instructions 

with nori-PV l^eachers. Thus, insignificant differences be-* 

' ^ ( ' ' ' S 

tween PVLand non-PV cla'ssrooms could indicate a high level . ' 

of interaction among PV and non-PV teachers. T.hird, two of 

the eleven models had no comparison dlassrpoms to be observed 

^Therefore, we decided to 'ignore non-PV scores and tio run all 

further .analyses with PV classroom data only. ^ 

-. * . • . 

Using PV classrooms only, then, we 3 an another least- 

squares analysis which provided F-tests for'*mcdel effects • 

and for site-within-model effects. This analysis also gave 

us t-values"for each model and for sites-within-models on 

A ^ ' \ ^ 

all ^variables . We recognize the problem of^miiltiple compar- 
?.sons in t>iis ana-lysis and intend to u'se thes's t-values merel 
as informal indicator3 of significance- Thus, with degrees 
of freedom close to. 100, an effect in this chapter *is signi- 
ficant at the .01 leyel if ^ its t^value exceeds 2*. 6 and sig- 
nificant at -the .001 level if its t-"value exceeds 3.4. 

We -also ran correlations with only the PV data oh all 



/ 



variables fall to fall, sprifig /to spring, and fall to spring. 
We used the fall t»o fall and spring to spring correlations ) 
to test hypotheses about relationship among variables^, i.e;. 



vf certain kinds of aduit behaviof correlated highly with 
certain kinds of cfcild behavior. We used th^ fall/^to. spring 
correlations to determine v/hether ,-scorear on a variable in 
the, fall correlated v/ith stores on ^that sam^^' varj.able in 
thQ- spring* Because these fall* to spring correlations were 
low on over a .third of the variables, and because we foun'd 
from the *t--value"s th^t-^model effects of ten ^changed ftom 
fall to spring, we -ran another exact least^squares analysis^ 
on difference scores (spring score mipus fall scote) to 'get' 
t-value.s for) model change effe^:t,s . We looked, too^^ 'at F- 
tests for changes- over all models from fall to spring on each 
of the variables. This analysis was helpful in^ indicating 
whether change over* all models on a variable was sig'^ificant, . 
and in showing; which' models changed significantly more than 

» e 

Others-- . * / 

With these analyses, we examined each" variable for sig- ' 

nificant model effects which could be interpreted as .actual 

differences among models. _ Roughly^ speaking , an effect for 

a model was "convincing'* if its -^-value was Significant at* 

tine .01 level and if site effects, appeared ' less important 

tKan the model effect. Ih other words, if it was clear thai " 

only one site in a model with tv;o or three ^ites scored higher 

than the grand mean on a variable and the other site ( s)*.. scored 
I . ^ 

v;ell below the grand me^n^it could hardly be said that that 

"model*' was significantly high. ' However /r - since we 'have five 
\ »/ • ^ ■ 

models with three sites, four models with two sites, and tv;o 



-models with only one,\site, the "site effect importance" ' 

criterion could not be applied rigidly. '•Throughout this 

♦ . . . " < . • 

chapter, we report site effects, where they .appe^.red impor- 

^ ^' . \ ^/ ...^ . ■ > • 

tant. 



We i 



nterpreted large f all-to-sprii-ig differences v;h^n 



an expl-anation 'seemed plausible and otherwise attributed 
.t'he change to observer effects and to natural day-torday 

■ • . • . . ■ J,, t 

chan<^e in ^ Head Start center. ^ . 

Thus, we us^d all 'of our data together to explore for 
."fTatterns of model effects.. The- F-tests' for model effects 
were significant at the' .001 level on almost all of our 
variables, but because site effects were also significant 
and becausfe there were fail to spring differences, we could 
not assume the existence of actua^ differences in^ practice 
among models from the sign^-f icance of the F-tests. • If fall- 
to-spring changes on a variable were sig* ificant, and if 
site variation was substantial, then we assumed the variable 
to Ibe of little value in finding differences m pract3„ce 
among mo^^ls. Although it is possibly to interpret site 
effects and possitJle to interpret fali to spring change, .it 
is not possible to find ponvincing model effects when both 
'sitfe effects - and f all-to-tspring chang*e occur together jdu the 
same ♦variable . The fallowing hypothetical taJples il^lustrate 
(1) fall to spring change .without site effects (which is in- 
iii^rpretabie) ; (2) site effects withou^ fall to spring change 



(which is also interpretable) ; and (3) site effects with 
fall to spring change (which, regrettably, is uninterpretable) 
We Ido not intend these hypothetical -^tables to represent actual 
variibles ia our analysis^ but to serve as "caricatures" of 
the ^prdbleiTiS involved in finding convincing model effects.. 



\ 



.HYPOTHETICAL TABLES 



SituatjLon 1,: Fall to Spring, change, without important site 
effects ' 



Fall 



Spring " 



Model 1 



Site A = 2 
Site B = 3 
Site* C = 4 



' Model 1 Mean 
Model 2' 



Site E = 6 
Site F = 3 
Si^e G = 6 



Model 2 Mean = 5 



Model 3 



Sitb H ,= 4 
Sit^ 1=5 
Site^ J = 3- 



Model 3 Mean = 4 



\ 



Site A =.5 
Site B i= .4 
Site C = 6- 



Model 1 Meah = 5 

Site E = 3'- 
■. Site"? = 2 
Site G = 4 



* Modfel 2 Mean = 3 

. Site'H = 6 
^ Site 1=7 

Site J = 5 



Model 3 Mean 

V 



lr\ Situatibri 1^ we see 'that model 2 is high in thd fall^ 
and that iuodei 1 is low. In the spring, 'however, model'' 3 is 

, y * -, * * . • ^ ■ 

high and model 2 is low. Although the ^^ites, did' not change 
at the same, rate (j^or example, in model 2, site fi decrecfs^d 
3 points, site F 1 point, and site G 2 pofnts) , all of the 
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sites in one model (hither increased or decrda. fron fall to 
spring. Thus ^ if this variable v;ere "children having tan-^ 
trums/'^ we could sciy that Models -1 and 3 showed an increase 



\ 



in* tantrum behavior and that model 2 shov/ed . a | decrease^^^ 1'his 
variable might . cdrfel^ate highly with other variables^ such 
as "adult negative ja'ehavior , " or it radght not be correlated';, 
with any other variable. In. any case, we can draw some; 
conclusions about differences among mo^^ls. 



., Situatioji 11: Important site' effects without fall to spring 

change 



Fall 



Spring 



Model 1 



Site A = 3r 
Site B = 2, 
Site C = 1 



\ 



Model 1 Mean = 2 



Model 2 



Site E = o 
Site F = 1 
Site G =, 3 



Model 2 Mean = 4 



Model 3 



Site H = 3 
Site 1=9 
Site J = 3 



Model 3 Mean = 5 



Site A = 2 
Site B = 3 
Site C = r 



'Model 1 Mean = 2- 

Site E 6 

, Site F ^ 2 

Site G =^ 4 



Model 2 Mean = 4- 

Site H = 3 
Site 1=8 
Site J = 4 



Model 3 Mean = 5 



Situation II shows model 3 consistently high and model 
1 9on-sistently low, with model 2 ' slightly above the grand 
meaiu However, site E in model 2 and site I in model 3 are 
the only sites substantially above the grand rr.ean* Thus, if 
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thisw variable were "chi ldren,.,asking que'stiari"^ ^we . can say 
that children ask fewer questions i"n model 1 than in ci^dels 
2 and 3, but that the higher scores in jnodels 2 and 3 are mainly 
due to high scores in sites E and I. , Although conclusians about 
model differences 'must , be weak with such large site to site 
variant ioni}^ at le^s't the consistency of the sQore.s indicates 
that -ther.e might be genuine' differences on this variable among 
sites , if not among models . = • . ■• 

Situation III: Important site effects with fall to spring chang^k^ 



Fall 



Spring 



Model 1 ' ' "Site A = 

.. ^ ■ Site- B = 

• . • Site C =' 

Model 1 ^ean = 

Mode^L 2 Site E = 

Site F = 

Site G = 



Model 3 




Mode^ 3 Mean 



2 
1 

3 
8 
3 



Model 2 IMean = 4- 



Site H p 3 
Site 1=9 
Site J = 3 



Site. A 




1 


Site B 




10 


Site Q 




4 


1 Mean 




5 


Site °E 




■2 


Site P 




■6 


Site G 




3 


• 

2 M^^an 




3 


Site H- 




• 2 


Site .1 




3 


Site "j 




7. 


3 Mean 




' 4 









Situation III illustrates the impossi 
convi|ncing model effects, when both site effects and- f all^to 



•I 

in^ ch 



spring change occur together on a variable. No site or nodel 
is. c^sistently higher than others. Although no variable in 
our actual .an/iysis looked quite hcltcr-fikcltcr as situation III, 
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. .this example certainiy shows that we' can dr^w no, conclusions' aKout 
differences aiT\ong models v/hen site effects and fall to sprinfr',,- ' 
change are significant on the same variable. , * 

Th the follo\^ing section, we discuss only those variables- 

'which indica,te convincing model effects. do not discuss '-^^^ . 

variables one through seven for two reasons. First, they 

represen-t the type of activity that observers- sej^ctpd for • 

their ^fivc -minute observations, and thus only indirectly 

suggest the f-requency of' occurrence of these activities in' 

"the classroom. Second, only one of these .feeven . reflects actual , 

W ■ 

differences ain.ong models and since thac one is' " acaderai c ' ,' 
activities' and we have- five other variables to describe academic 
activity, we decided t^ disregard -.the first seven as a group. 
• Also, we do riot discuss the few variables which showed 



_insignificarit ,F-i^6sts for model effects. .Neither do we discuss 
^ some variables which are combinations of other vap.able3; " 

e.g., we disregard "adult praise and acknowledgment" but rather' 
, "deal with "adult praise" and "adult acknowl-dgmecit separately. 
Although ue recogni.ze that dj^egarding' variables is a rather 
dangerous procedure, we feel fairly confident that we have 



included those variables vJhich show gejiuine differences in 



practfl'cos among inodels 




^DllSCUSSION OF VARIABLES \ 

According to 6ur\s^rate%y , twenty-two of the total 
fifty-one variables indicate substantial differences among 
ncae Is. These twenty- two are divided i^to three groups • 
for purposes' bf discussion: academic activity , iridependent 
child 'activity, and adult/child interactions. Along, with 
the discussion of each ^variable is a table shpwing mean-^ 
scores for each model. ' At 'the end of each of the three 
group^s^ are summary- tables showing which models are 
signi:^icantly high'^or low on. those measures. In Appendix G 
are tables of means, standard deviations, and sample size 
for each model ^fT all /fifty-one variables. 

I. Academic Aqtivity : 

i ^ " . • 

academic activity (frequency of occurrence) 

:"- adults with chilaren in academic activities 

aide ' s 'participation in academic activ'i ties 

teachers and' volj^ti^ers ■ wifth phildren ini^ 

academic activities , 

42 : independent children' in academic activities 

y 

AcademicL- activity isl defined a^ Activity C and Activity D 
(see . variables 3 anci 4 in Appendix fJ, ' "^cadeinics*" do not 
nclude games and puzzles, but. do include nuinbers, alphabet, 
and "scienqe" acti\^ties. It is interesting that none of the ^ 
above variables changes sicni f icantly over all ^nodels from fall 
to spring. 'Although variables '9 and 42. show ..some significant 
within model changes from fall tc^ spring^ txhis group of vari- 
abl-ts&l 3Gems to represent one the more stable erlements in 



Var 9 
'.S 
13 
43 



classroom activity 



Further, site effects are'' less marked on academic activity 
variaJoles than on ^ny other group. Clearly, then, evidence 



of significant model effects here reflect actual differ- 

' .... 
eRces in practice among models. • ' ' 

Var 9: -academic activity (frequency of occurrence) 



The table on variable 9 shows Bank Street, and Kansas 
significantly high in the fall and Oregon Kansas -and Pitts- 
burgh high in the spring^ The fall Bank Street score is^mis- 



i 



leading because onl^ th^ Wilmington site is above the grand. * 
mean, while the other two feank Street sites are well below. 
Wilmington is still. high in the spring, but not high enough 
'for Bank Street's scope to peach 'signi ficance . 

Pittsburgh's high spring score is puzzling:" Pittsburgh . 
has only one site and we cannot guess .whether the fall 
observer- saw more " typical " " activity than, the spring observer. 
We might assume given that Pittsburgh v/as new to Planned . 

' - ' ^ y ' ' ' • 1 

Vo^rdation in the fall of 19 70 and that the model emphasizes ' 
acadc;\iic activity, that the mod^el shows /better implementation j 
in the spring. 

The Oregon and Kansas scores show strong evidence of a I 
great deal of academic activity in their 'classrgoms . Site 
effects are uni^iportant ; fall' to spring change's, are within th^ 
limits of normal fluctuation; and both models .stress academifc 
iearnincT in their classroom theory. ^ Thus, it is fairly certiain 
that children enrolled in Ore.gon or Kansas Head Start programs 
will be involved in more ac^^S^mic activity than ciiildren attend- 
ing othpr .Planned Variation programs. 



Dtn^r 
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Mear Sc o res by Mode l on- classroom Obse rvati on 

Var^. 9. Academic Activity (Frequency of Occurrence 



/ 





r d X X 


# oprmg 


Far West 


V # J U D 


U . 4 Z J 


f 

U." of Ariz. 


0 484 


n 717 

U . / X / 


Bank St. 


1 n ^ Q ' . 


1 n 7 Q c; 


U. of Oregon 


0.969 


1.194"^"^ 


U.* of Kansas 


1. 220++ 


1.164 + + 


Wi/Scope 

• • 


0,207" 


0.225-- 


U. of Florida 


0.468 ' 


< 

0.197"" 


EDC 


0.323 


0.458" 


Pittsburgh 


0.'807 


1.527"^"^ 


BEC 


0.757 


0. 749 


Enablers 


0,30 5" 


0.412" 



Grdnd Mean 



0. 634 



0.665 



: In the ct..'Ove table + indicates significantly ^ligh at 
.01 lovel (t value = 2.6/br ever); +'+ indicates siqni fiv.cn cly 

• sionificantiy lov; at . 01 'level ; — indicates signi f icantlv 
low at .001 level. • . " 



ERIC 



Var 8: ^adults with children in acaQenlic act^'.vities 

, . ' V *" ."^ ' \ 

Variable 8 reaffirms pur earlier statement about Oregon 

and -^Cansas classroom^ and shows that most of the acad^ic 
activity occurring there includes , both adults and children. 
The score? on variaole 8 for the Oregon and 'Kansas models 
are very similar to their scores on var 9, indicating that 
almost no academic activity occurs that is* not directed, 
or at least observed, by adults. In contrast, Pittsburcjh ' $ 
high spring score on varicible 9 is not repeated on variable 8, 
indicating that a high percentage of P ittsburgh-' s academic 
activity does not include. both adults and children. We will 
investigate this hypothesis later' with variable 42 (independent 
'children in academic activllfcy) . , * " 

Var 13: ai-de ' s participation in academic activitifewS t 

■ ' On variable 13, v;e see that the Kansas model, remains 
hiali, indeed high enou>gh to indicate 'that aides participate » 
in more than half of the total academic activity in the 
classrooms (see variable 9). The Oregon model is only 
slightly higher than the grand mean in the^ lall and although it 
is significantly higher than the grand mean in the spring, 
only one of their sites "(t^. Las Vegas) is high, "the other site 
^Tupelo) J z quite low on this variable both in the fall and 
in the sprinq. Thfe, because of large site effects in the 
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\' *Mean Scores by Model on Classrcom Observation 
Var, 8. Adiilts with Children in Academic Activities 



Fall 



Spring 



'F-ar West 
U. of Ariz. 
Bank St. 
♦U. of Oregon 
U. of Kansas 
ili/Scope 
U. of Florida 
EDC 

Pittsburgh 
REC. 

•Enabler s 
Grand Mean 



\ 



0.209 

0.349 

0..430 

0.936+ 

1.206++ V 

0.1S7-— 

0.419 ■ • 

0.24 2 

0.542 

0.633 

0'.4 76 



0.228 

0.471 

0.402 

1.126++ 

1.124++ 

0.174 

0.117 — 

0.301 

0.649 ■ 

6.482 

0. 295 

0.483- 



Note : In the above tabl'e + indicates significantly high at 
.01 level (t value = 2.6' or over) ; .++ indicates significantly 
high at .001 level (t value = 3.4 or over) . - indicates 
significantly lov.' at .01 level; — i-ndicates significantly 
,low at .001 level.- • ' > 



Vean Scor,cs -by f'odGl-on Classroom Observation 
• ^ 

Var. 13. Aide's Participation in Academic Activities 





Fall 


■Spririg 


Far West 


.'0, 059 


0.087 


U. of Ariz.' 


0.097 


0.195 


Bank St. 


0.154 


0.111 


U. of Oregon - 


0.242 


0. 378+ 


U. of Kansas 


" u.791^"*" 


0. 77 6"'"+ 


Hi/Scope' 


0.032" 


0.068 


U. of Florida 
EDC 


0. 170 

f 

0.0-87 


0.046 
0.093 


Pittsburgh 


0.148 


* 0.207. 


REC 


0-,252 


0. 021 


Enablers 


0.06:. 


0.077 


Grand Mean 


0.201 ; 


0.204 ' 

« 



Note : In the above tcible + indipates significantly high at 
.01 level (t value = 2.6 or over); i-+ indicates significantly 
high at .001 level^ (t value = 3.4 or over). - indicates 
signifit:a:itly low at .01 level; — indicates significantly'.' 
low at . 0 0 1 le ve 1 . 
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Oregon model, we conclude that (5nly the ^Kansas model is 
significantly high on variable 13. 



Var 43: teachers and volunteers with children in . 

academic" activities ^ 



.Variable 43 provides a dis^tjicnct contrast to variable 13. 
This variable shows Oregon significantly high (at ,001 level] 
on both observations and Kansas slightly higher than the gr'and 
mean. Thus, we*ha.ve a further Refinement of variable 9: more 
thah half of the Oregon model ' s academic activity occurs 
with teachers and^or volunteers, while more than half of the 
Kansas model's occurs with aides. 

Var 42: independent children in academic activities- 

Vai}^cible '42 completes the academic activity spectrum. 
Here the Oregon and Kansas models are low, as we might 
have predicted from their high scores on adults with children 
in academic activities. Bank Street is significantly high, 
but' as v;ith variable 9, only the Wilmington seore is above 
the grand mean. The Pittsburgh model is high in the spring, 
indicating that^,^like the Wilmington site in the Bank Street* 
model, a large proporti'on of their academic activity occurs 
without adult direct supervision. 
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■'lean, Scored" oy '^odol on Classroom Cbservatj.oh 
Var, 43. Teachers and Volunteers ' with Children in 



Academic 


Activities 






Fall 


Spring 


Far West 


0.150 


0.141- 


U* of Ariz. 


0. 252 


0.276 ■ 


Bank St. . 


0_. 276 


0.291 


U. of Oregon 

U. at Kansas ^ 


0.694'^'^ 
0. 415 


0.7'4 8'^'^ 
0.349 


Hi/Scope 


0.135- 


0.106 — 


U. of Florida 


0.249 


. 0*. 071 — 


EDC 


0.155 


0.208 


Pittsburgh 


0.394 


0.442 


REC ' 


0.380 


0.461 


Enabler s 


.0.14 5 


0.218 


(■ 

Grand Me^n 


0.275 


0.278 



Note : In the above table + indicates signi ^iceoitly high at 
.01 level (t value = 2*6 or over) ; ++ indifcates significantly 
high at .001 level (t value = 3.4 or over). - indicates 
significantly low at .01 level; igridicates significantly 

low at .001 level. - ^ 
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Mean Scores by Model on CI a ssroom ObservatS 
yar,.42. Independent Childreh in Academic Activities 





Fall 


Spring 


Far West 


0.297 . 


0^96 


U. of Ariz. 


0.135 


0.-245. 


Bank St. 


. ^ 0.599"^"^ 


0.383+ 


U. of Oregon 


• 0.033 


0.068 


U. of Kansas 


0.014 


0 . 04 0"" 


Hi/Scope. 


0.040 


. 0.051" 


U. of Florida 


0.'049 


0.079 


EDC 


0 .081 . - , 


0.157 


Pittsburgh 


■0.2'65 


0.878"*""'' 


R^C 


0.124 


0.267 .. 


Enablers 


.0.0 99 


0,117 


Grand Mean 


0.158 


0. 182. 



Note : In the above table + indicates significantly high at 
• 01 level (t value 2.6 or overX; ++ indicates significantly 
high at .00.1 level .'(t value = 3.4 or over)". - indicates 
signrf ioantly low at .01 level; — indicates -signi ficantly 
low at .001 levels ' ' 



TABLE 33 



^First Sun\mary Table 





Academic 
Activity 


Adults wth. 
Childrn. in 
Acad . Acts . 


Aide ' s par-| 
ticiptn,. .in \ 
Acad . A'cts . 


Tchs^& volsi Indopond; 
wth. childrnj childrn. 
in Ac'.Acts.i in Ac. Acts. 




Fall 


Spr. 1 


Fall ) 


Spr. 1 


FalJ 


Spr. ■ ^ 


Fall [ 




Fall 1 


Spr .. 


Far West 


• 














- 






IT O'F AT*i7 






















Bank St."^ 


+ 




















U. of Oreg. 




++ 


+ 


++ 






++ 


++ 






-U. of Kan. 




++ 


++ 


• • 




+ + 










Hi/Scope 








/ 














U. of Fla. 


/ 

/ 

/ 

/ 

/ 

/ 

/ 




















EDC n , 

^ Pittsburgh 




















REC ' 




















Enablers 








* 






f 









/ 

o 

Note : Ih the above table + indicates significantly high at .01 
level (t value = 2^6 or over); >+ indicates significantly high % 
at .001 level (t value = 3.4 or over). - indicates significantly 
low at .01 level; indicates significantly low at .001 level. 



I 

I 
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II. Independent Child Activity; 



independent children in academic activities 
indepe ident child activity 

independent children in non-acadGinic activities 

wide variety of activities 

child self-learning with concrete' objects 



Var 42 
10 
44- 
11 
23 

iTh^ evidence which differentiates niodels on this group 

of variables is less clcar-^cut than on academic activities 
va>:iables. IVo of the five variables (10 and 23) in^the 
independent act! vi'ty group change si^grii f leant ly from fall to 
spring /:^Ver all models. However, all cf these changes are 

positive ,i)indicating, perhaps, that children who are more 

\ ' " ' ' . ' ■■ ■ . 

familiar v/ith their classroom environment are more likely' 

to behaye indepehdentlv . > Within-model significant chancres 

are comparati vely ew . Thus, f all-toVspring differences 

present only a minor difficulty. ^ ^ 

Site effects, on the other v;ords are large-^. especially 

■in the Bank Street and Far, West models. One would expect 

more 'site effects on this group of variables than , on academic 

activity variables, since, 'as anyone v/ho has ever worked with 

preschool children knows, it is Often difficult to "structure" 

independent arctivity. Although adults in the classroom 

can encourage and reward independent activity, they cannot 

"schedule" it in the same way that they can ^"schedule " more 

structured academe .activity. Further, the level of indepen-- 

dent activity may be influenced by factCj^s outside the control 

of teachers in the classro^Ti. ■ For example, independent acti- . 



c 9 

\ ■ 



vity might be higher jusi; before holidays when ch-ildrcn 
are keyed up, or when new toys or equipir^ent are introduccc 

r - 

into the classroom. 

^ For these reasons, then, the requir-ements .for signi- ' 

ffxrant model differences in this area are somewhat less 
stringent than in other areas.- • " 

Variable 42 was discussed in the previous section on' 
acaflemic activity , and shows- mainly that the ^ Wi Imington site 
in the Bank" Street model is k^otstandingly high, and that Pitt 
burgh is signi/icantly high in the spring. 

Var 10:- independent child ^ctivi.ty ^ ' . 

The table on variable. 10 indicates that the Bank Street 
model is. signi'f icantly --high in both fall and spring observa- 
tions. However, 'so;ne* si.te effects 'confuse t:his finding. In 
the fall, Wilmington's mean is. far higher than any other site 

,thus making Bank ' Street ' s mean^ signifidantly high. The other 
^ - " ' . 

two sites tBo-ulder arid Elmira) are both near t^^e grand mean. 
In the spring, the Wilmington mean- is lower than in the fall, 
but'Elmira's mean is higher. Boulder is stillnear tne grand 
mean. Thus, site effects, although Xess evident in the sp^rin 
are important in' the Bank S;treet rtiodel, ^! 

The Pittsburgh model is significantly high only in the 
spiling. As on, two of the acade!nic activity variables, this 
model's scores rise significantly from fall to spring. ' Since 
the Pittsburgh model was new to Planned Variatior. in the . fall 
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Mean Scores by M o del on /C lassroom^^Ob servation 
Var. 10. Independent Child Activity 



^ 



Fall 



Spring 



Far. West 
U. of ^riz. 
Bank St. 
U. of Oregon 
U. of Kansas 
Hi/Scope 
U. of Florida 
EDC ' ^ ^ 
Pittsburgh 
REC 

Enablers 



1. 656 

1. 380 

2. 654^''' 
0^.2 4 9" 
0. 561 
0. 720 
0.'*649 

0. 843 

1. 244 
1. 484 

i.ioS 



1. 461 - 

1. 264 

^.450++ 

0.677 

0.550~- 

0.998 

0.92? , 

1.138 

2.454+ 

1.532 

1.536 



Grand Mean 



1. 141 



1.296 



1 

Fall to Spring change over all models 



N ote ; .In the above t^le + indicates significantly high at 
.01 level (t value = 2.6 or over); 4-+ indicates significantly 
high at .001 l^vel (t value, = 3.4 or over). - indicates 
significantly low at .01 level; indicates signi ficantly 

low at .001 iGvel. 



r 
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^ / 

of 1970, we /ttach more importance to its spring scores. 

Var 44: independent children* in non--academic activities 

Variable 44 shows Bank Street again significantly hid^h, 
but with the same ^type of site effects that oci:^urred on 
variable 10. ^ 

... ) 

Var 11: wide variety of activities 

■ .' . ^ ' ' 

We include this variable in the independent activity 



group because it correja.tas so highly with variable 10 "(.899 
in the fall; . 849 in the spring)'. Bank Street is high, but ^ 
with the Wilmington site! again f^r higher than any other site 
in the analysis. Bouirlier and Elmira are s'lightly above the • ^r^^ 
grand mean on both fall and spring observations. 

The ract that the Florida model is low on both observa- 
tions may be the result of that 'model's enjphasis on parentj ^ 
instruction rather than classroom activi^ty. . \ 

Var 23:. Child self-learning with concrete objects 



Vai^iabl'e 23 is somewhat confusing: - the variable 
includes three different observer codes: the code for respond, 
the code for comment, p^ay, and th(2 code for -"teach , inform.. * . 
We are unsure v;hethGr d child playing with a^ trruck or dolls 
would be- included in this variable ©r whether "concrete ob- 
|| ^ jects" refer only to learning \ools , such as Cuisenaire rods. 



t 
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Mean Scores Model on Classroom Observation ^ 
VaJ. '44. Independent Children in Non --Academic /ctivitic/s 





Fall 


Spring. 


V ■ ■ ■ 

Far West 


358 

1.245:^ 


1.265 


U. of Arizona 


1. 018 


Bank St. 


++ 

2.054 

y 


2. 066++ 


U. of Oregon 


0.217" 


0. 609 


U. of Kansas < 


0.547 


0.511"' 


'Hi/Scope 


0. 680 . 


0.947 ' 


U. of Florida 
EDC 


0.600" \ . , 
0.761 


0 . 843 
0.981 


Pi ttsburgh 


0.979 


1.57-6 


REC 


1. 360 


, 1.265 


Enablers 


1.008 


1.419. 


Grand Mean 


0.983- 


1.113 



Note : In the above table + indSfcat^3 significantly high' at 
.pi level (t value = 2.6 or over); ++ indicaterv sicnif icantly 
high at .001 level (t value = 3.4 or over). - indicates 
significantly lov; at .01' level; indicates significantly 
lov; at .001 lev^l. ^ ' . ^ 



J 
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Mean Scores by Model on Classroom Observation 



Var. 11. Wide Variety of Activities 



Fall 



Spring 



Far West 

U. of Ariz. 

Bank St. 

U. of Oregon 

U. ,of Kansas 

Hi/Scope 

U. of Florida 

EDO 

Pittsburgh 
REC 

Enablers 



2.605 
1.987 

i..4y- 

2.019 ' 
1. 791 
1.553 
1.647 
1 .,8'3 5 ' 
2.050 
1. 860 



2. 361 
1.91*5 



3.786 
i. 897 
1.615 
. 1. 935 
l'.5-59 

1. 758 

2. 65.8 
2\ 180 

•2. -17 3 



++ 



Gra,nd Mean 



2^112 




2.154 . 



Note : - In tjiS above table + .indicates signi-ficantly high' at 
.01 level 'Tt value = 2.i or over); indicates significantly 
hiqh at .001 level (t va^ue = 3.4 or over). -^indicates 

signi f^q^^^^ntly low at .01 leve-1; indicates significantly 

low at .001 level. 
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• Mean Scores by ^^ ^del o n Classroom Observation 
Var. 23 Child Sel f -Lfearning' with Concrete Objects 





Fall 


Spring 


— \: — ^ 

Far West'- 


1.841+ 


2. 914"^"*' 


U . of Ari^ • 


0.998 


1.511 


Bank St, 


1.563+ 


1.156' 


U. of Oregon 


0.0 


0.552 


U. of Kansas 


0 . 0 ->U 


i . 0 0 D 


Hi/Sco^)e 


0.111 


0.052 


U. of Flo*rida 


0.216' 


0.603 


EDC 


0.839 


0 . 484 


Pittsburgh 


OjfO 


1.494 


REC 


0.835 


1.564 


Enablers^ 


0,031 


0.071 


Grand Mean- 


0.578 


0.910 


Fall to Spring 


change over 


all models . 



Note ; In the aib'ove table + indicates significantly high at 
.01 level' (t value = 2.6 or over); ++ iriSicates significantly 
high at .001, leve.l (t value =^3.4 or over)-. - indicates' 
sigriificantly lov; at .01 level; — indicates ^igni ficantly 
low at .001 level, . . ^ 
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* TABLE 34 

Second' Summary Table 





! 

Ind. • Child I 
Activity \ 


Independ . 
Childrn in i 
Acad.* Acts. 


Ind. Chid- 
rn. in'\non- 
Acad . /Acts . | 


Wide Var- 
iety of 
Activities 


Child self- 
learng wth 
concrete obj 




Fallj 


Spr. . 


Fall . 


SpT . 


Fall . 




Fall , 


Spr. 


Fall . 


Spr. 


• 

Far West 












1 

• 






+ 


++ 


U. of Ariz. 




■ 


















Bank St. 


++ 


++ 




++ 






++ 


++ 


+ • ■ 


























U. of Kan. 








• 














Hi/Scope 






















U... of Fla. 








i 














EDC 








i ■ 

i 

• 














Pittsburgh 






















REC 








1 
1 

1 














Enablers 








• 

1 















Note : ^ In the above table + indicates significantly H'igh at 
.01 level (t .value = 2.6 oflCpver) ; ++ indicates significantly 
high at .001 level (t value ^ 3.4 or over). - indicates 
significantly low at .01^ level; indicates significantly., 
low at .001 level. 
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We tend to assume the latter^ since the scores dd not clearly 

resernble other independent activity scores. 

V^ith this confusion in mind, then, we see that the Far 

West model is significantly high in both the fall and spring. 

Site e'ffects are important heie, since the Tacoma site is 

below the grand mean. Duluth is far above the grand Hie n , ' ' 

as is iBoulder in the Bank Street model. Thus, Duluth and ^ 

Boulder are the only two sites in the analysis that are con- 

sistently high. Given Far We^^t's ^strong emphasis on "auto-^ 

telic'' (self-rewarding) activities, we conclude that the 

J 

Duluth site is well implemented in this area. 

III. C hild/Adult Interactions * 

A. Adult focus on groups of children 

Var 12: adult interactions with one or two children 
29: adult communication focus: one child 
30: adult communication focus:, small group 

This group of variables provides information about 

how many children an" adult in the classroom addresses at one 

time. Since tKe'se variables represent adult behavior rather 

than child behavior and since models are implemented through 

the adults in the classroom, we would expect these variables 

to show important moael effec^^sTV 

■ ■ \ ■ 

Var' 12: , adult interactAgns with one or two children 

- ■ ■■■ ' . 

Vax-i£iblG 12 shov;s Pittsburgh and Bank Street coxlsistentl 
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Mean Scores by Model on Cl-assroom Observation^ 



Var. 12. Adult Interactions With One or Two Children 





Fall 


Spring 


Far West 


6. 320 


0.362 


U. of Ariz. 


0. 205 


0. 212 


Bank St. 


0.4 8.0^ ; 


0. 936++' 


U. of Oregon 


0.090"" 


0.102 


U. .of Kansas 


0.267 


0.081- 


Hi/Scope 


0.187 


0.. 307 


U. of Florida 


0. 140-- 


0. 208 


EDC 


0.151 


0. 278 


Pittsburgh , 


0. 683++ 


0. 968++ 


REC 


0 .342 


,0.429 


Enablers 


0; 243 


0. 203 


Grand, Mean 


0. 266 


0.340 


Fall to Spring 


change over all 


models 



In the above table + indicates significantly high at 



Note 

.O'l level (t value = 2.6 or over); + indicates significantly 
high at ,001 level (t value = 3.4 or over). - indicates 
significantly low at .01 level; indicates significantly 
low at .001 level. 
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higher than .other models. There are some site effects 
in the Bank Street model, however; Elmira is quite low 
in 'the fall, but that score rises in the spring indeed 
enough to give Bank- Street a significant positive change 



from fall to spring. 



\ 



Var 29: adult commun 



Lcation focus : one child 



The table indicates t&at Bank Street is low., which is' 
puzzling in view of its hidh score on variable 12. Either 
this discrepancy is anomalolus or Bank Street adults often .-^ 
interact with two children J but rarely ' with one "hild. 

The Kansas model is quite l^igh, with small differences 
among its sites. This high score is probably related to 
the system of rewarding children's behavior with tokens. 
An adult gives a child a token and fJraises him verbally, and 
thus "commui'iicates " with one child. 

The Hi/Scope model is significantly high only in the 
fall. In\this mp'del , the Fort Walton Beach site dropped 
from a mean of ' 13.012 in the* fall to 3.400 in the spring, 
thus giving that model a significant negative change, from 
fall to spring. It is interesting that, over all models, ^ 
adults communicated with one child significantly more in 
the fall than in the Spring. Perhaps this finding is re- 
lated to a significant positive change in child independent 
activity. Perhaps ghildren need less "rewarding" or "reminding" 
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Mean Scores by Model on Classroom Observation 



Var. 29. Adult Communication Focus: One Child 





rail 


Spring 


rar west: 


1 U . 4 U o 


9.590 


U* CJX riXTXZ* 


11 Q 
11. y D O 


Q Q A 1 






o . 2 55 


U. of Oregon 


12.755 


10.519 


U. of Kansas 


15.777+ 


15.763 






1 ri TOT 


U. of Florida 


8.941 


7. 321 


EDC 


9. 695 


10.112 

o 


Pittsburgh 


16.687 


12.778 


REG 


13.726 


14.224 


Enablers 


13.359 


10.69 5 


Grand Mean 


12.560 


10.798 


Fall to Spring 


change over all 


models ~~. 



++ 



Note : In the above table + indicates significantly high at 
.01 level (t value = 2.6 or over); ++. indicates significantly 
high at >001 level (t value = 3.4 or over)* - indicates 
significantly low at .01 level; indi ::r:tG3 3i<jni ficanujy 
low at. 001 level. « ' 
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in the spring': ^ 

Var 30: adult cominunication focus: ::inall group 
The Oregon model shows very high scores on this vari- 
able, scores which are probably related to their emphasis 
on small-group academic instruction. Because Oregon has low 
scores on variable 29 (adult communication focus:, one child) 
and on variable 10 (independen,*? child activity) we conclude 
that children in this model spend a great deal of time with 
adults in^^-Si?rall groups- ^ ^ 

Bank Street is low on this variable, possibly as a result of 
its stress on independent activity and on child interactions* 

B, Adult attention (responses) to children 

Var 22: adult acknowledgement to child 

33: adult positive corrective feedback 
46: adult praise of children 

These * variables represent three different ways of res- 
ponding to a child^s behavior. Acknowledgement is coiTin^enting 
in a more or less neutral way: ^'You're drawirig a tree, 
Jimmy." Acknowledgement can also be non-verbal, such as 
smiling or nodding at a child. Positive corrective feedback is 
a method of maintaining discipline or correcting . mis takes by * 
suggesting an alternative, by giving a reason why behavior is 
inappropriate, by praising :inother child *s more appropriate 
behavior, or by questioning the behavior. Examples: "Why ^ 
don * t you use these blocks instead of the ones that Rachel is 
playing v;ith?" "If you tear tne book, we won't be able to read 
it again/" '"Look at the v;ay 'Henry is cleaning his paintbrushes." 
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Kean Scores by Model on Classroom Observation 

Vaf. 30. Adult Communication Focus: Small Group 

1 



s 


Fall 


Spring 


Far West 


, ,2.339 


3.202 


U. of Ariz. 


2.016 


2. 252 


Bank St. 


1. 233'' 


1.732 


u . ox uregon 


i z . o4 U 


1 / . 131 


U. of Kansas 


3.569 


1 . 890 


Hi/Scope 


2. 088 


2.326 


U. of Florida 


2.706 


2. 554 


EDC 


3. 890 


3.421 


Pittsburgh 


3.377 


2.835 


REC 


1. 941 


2. 234 


Enablers 


2.275 


3. 424 


Grand Mean 


3.277 . 


3.785 



^^ote : In the above table + indicates significantly high at 
.or level (t value = 2.6 or over); ++ indicates, significantly 
high at .001 level (t value 3. .4 or over). = indicates 
significantly low at .01 level;' — indrcates significantly 
low at .001 level. 
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TABLE 35 



Third Sumniary Table 





Adult Inter- 


Adult Conununi- 


Adult Communi- 




actions with 


cation focus : 


cation .focus ;, 


A. 


one or two 


one 


small 




children 


child 


group 




Fall Sprijig 


Fall Spring 


Fall Spring 




Far West 
U. of "ArizQ/ia 

Bank St- , + \ ++ 

U. of Oregon , ++ 

U- of Kansas - + 

Hi/Scope ++ 
U. of Florida 
EDC 

Pittsburgh ++ ' ++• 

REC 

Enablers * 




Note ; In the above table + indicates siani f icantly high at 
• 01 level (t value = 2.6 or over) ; indicates significai^^ 
high at .001 level (t value = 3,4 or o.ver) . - indicates 
significantly low at .01 level; — indicates significantly 
low at .001 level. 
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"Are you sure you want to, stand on the table?" ^Praise is, " 

of course, complimenting a child's behavior. 

All three .variables changed negatively from fall to 

spring, although the changes on variables 22 and 2 3 were 

not significant. ^ We again suggest ^ our hypothesis th^it a,^ * 

chil.dren and adults grow .more comfortable with one another, ' 

^ mm ^ / 

there is less reason for adult behavior which rewards or alters 

lildrert's behavior. 




Var 22 : adult acknowledgement to. child 



On variable -22,1 REC is consistently high, with' B.ank 
Street low. Hi/Sdbpe and Enablers^.are significantly high' 
only "in the fall an^J^^rffgon high qnly in. -the sorina. This 
variable is somewhat mysterj.ous: although we understand 
acknowledgement to denote neutral ati^tention , we wonder whether 
calli-ng the roll would constitute an^ acknowle^Sgement ' of each 
child. -Since we have no informc:^tion as to th-e content of this 
type of communication, we take'^he 'above scores -at fpce value,, 
and use this variable in conjunction with the next' two, in order 
to find some characteristic differences among models *bn the type 
of attention adults 'give to children-. 

. Var 33: adult positive corrective feedback ^' - — 

From the variable 3 3 tablc:^, we see that no score is signifi- 
cantly higher than the grand mean in the fall, b,ut- that Bank 
Street, Oregon, and Florida are significantly lev;. - In the ' • 



ERIC 



-236- 



Mean Scores by Model on Classroom Observation 



Var, 22 Adult Acknowledgement to Child 





Fall 


Spring 


Far West ' 


0 . 6 8,4 


0.628 


U. of Ariz. 


1. 097 


0. 984 


Bank St. 


0.241"" 


0.452 


U. of Oregon 


0.592 ' 


'2.076^ 


U. of Kansas * 


1.355 , 


1. 007 


'Hi/Scope 


2.581++ 


. 0.993 


U. of Florida 


0 . 357-- 


0.648 


EDC 


0.694 


0.876 


Pittsburgh 


1. 822 


1. 985 


REC 


2. 515^ ^ 


" 3.*207++ 


Enablers 


1. 983+ 

< 


V:.. 1.82 5 


Grand Mean 


1. 263 


1.179 



Note : In the above* table + indicates significantly high at 
.01 level (t value = 2.6 or over); ++ indicates signi.f icantly 
high at .001 level (t value 3.4 or over). - *irqci^icate« 
significantly low at .01 level; indicates sign:/ficantlv\ 

low at . .001 lev.el. 



I 
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J 



Mean Sco res bv Model on Classroom Observatio'h 

\ 



Var. 33. Adult Positive* Corrective P'eedback 



Fall Spring 



Far West 1. 099 0.282 

U. of Ariz. - i.510 ' -1.216 

Bank St. ' ' 0.193"" 0.415 

U. of Oregon " 0.420 — 0.195 

U. of Kansas 1.348- 1.507++ 

Hi/Scope 1.276 \ rO-286 

U. of Flori^da 0. 607" 0.855 

EDC 0.610 0.513 

Pittsburgh ■ 1.577 . 0.510 

REC 1 . 159 . 1. 272 

. Enablers 'D.881 ' i.'336"^ 



Grand Mean 01947 0.759 



Note : In* the above tabic + indicates significantly high 

at .01 leveT (t vaiud = 2.6 on over); indicates significantly 

high at .001 level (t value 3.4 or over). - indica4:es signi-' 

ficantly low 'at *01 levc^l; indicates s,igH(if icantly lov; at 

.001 level. " . x--^ * ' " - " 



r 
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spring, Kansas and Enablers are high, but nc/^core is 
significantly low. There is. a great deal of fall spring 
change on this variable, with Far West, Hi/Scope, -i^nd Pitts- 
burgh shov/ing substantially less corrective feedb^k in the 
spring. Vie surmise that the changes in these models indicate 

more relaxed adult behavior in the spring. , 



The Kansas and REC models ,. on^he other hand, show slight 
increases m corrective^ ■ feedback m the spring, and thus, 
these two models are-- distinctive in the consistcnt]^y High 
frequency of adult * corre ctive feedback in -their classrooms. 

Var 46: adult praise of children 

Here, Oregon, Kansas, and Pittsburgh are' all consistently 
high. Oregon's mean is lower in the spring, and compared 
with a higher spring score on acknowledgement, we see that 

whereas Oregon tdachers expressed more praise than acknow- 
ledgement in the fall, they expressed approximately eq^al 
y amounts of ^p^raise and acknowledgement in the spring. Thus, 
the Oregon model also follovzs the^attern of ' more relaxed 
adult behavior in the spring. . Kansas is high on both 
corrective feedback and praise, and just ^s we related their 
h'igh score on adult coininunication focus: 'one chiJ.d to their 

system of awarding tokens for good, behavior, we also relate 

\ 

their scores on feedback and praise to this system. 

Pittsburgh is high on faoitoack^in the fall, and on praise 

in both fall and spring. PittslMirgh, therefore, is outstanding 

in aiuount of attention uiven to one or two children at a time, 

since they are high on tv/o "attention" variables, significantly 

» 
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MeAn Scores by Model on Classroom Observation 



/Var.- 46. 

/ 

1 

/ 


^Qult Praise 
Fall 


of Children 

Spring 


Far West 




0. 690~" 


0.53^ 


U. of Ariz. 




0. 529"" 


■ 0.552 


Bank St. 




0. 298'>" 


■ 0. 310-- 


U. of Oregon 




3. 614++ 


2. 137+ 


U. of Kansas 




3.141++ 


2.615++ 


Hi/Scope 




1. 819 


1.014 


U. of Florida 




1. 052" 


0. 506" 


EDC 




0.876- 


0. 999 


Pittsburgh ' 




5. 080++ 


3.884++ 


REC 




0.634- 


0. 595 


Enablers 




0.676 


1.35 5 


Grand Mean 




1. 540 


1.249 



ERIC 
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! 



Fall tc Spring cYidiXxoOi over all models 



Note : In t^e above table + ipdicates signi ficantly high at 
.O'J level (t value ^ 2.6 or over); •f+ indicates signifi- 
' cantly high at .001 level (t value = 3,4 or over)'. - indicate: 
siqni ficantly low at -.01 level;. indicates significantly 
' lev; at .001 level. ' . 
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TABLE 35 

T 

Third Summary Table ^ - • 



B. 


Adu^Lt Acknow- 
ledgment to 
child 


Adult Positive 
Corrective 
Feedback 


AdUlt Praise 

of • 
^^Chi Idren 








Fall Spring 


Fcill , Spring 


Py] .1 Spr j ng ' 






Far West 












U.' of ^rizona 












Bank St- 












of Orego?! 


+ 










U,^ of Kansas 


♦ 


' ++ 








Hi/Scopfe 












ofi Florida 










< 


EDC 













Pittsburgh ++ ++ 

REC , + ++ - ,■ 



Enablers + + 



Note: In the aLove table + indicates significantly high at 
TOl level (t value = 2.6 or over); +-f inidcates signi f igant.ly 
high at .001 level (t value - 3.4 or over). - 'indicates sig- 
nificantly low at .01 level; — indicates significantly low at 
.001 vlevei. ' ' ■ ^ . 
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4 

high on both adult interactions wi throne or two children, and 
above the grand mean on adult coimnunication focus: one child. 



Questioning 

Var 15 
16 
18 
20 
27 



adult direct questionitig of child 
'child response to adult direct question 
adult feedback to chi Id response , ' 
\adult asking thought questions ^ 
child asking quGstion.5 



Vie v;ill deal -with the first three vari ables in this 

group togethe^r/ as they are a "process" which is imoortant 

in the Oregon, Kansas, IPX and REC models.' These models 

*' . • 

have "prescribed learning" activit^^ as part of their model 
specifications. In other words, these models attempt to 
teach children specific skills in a structu:|^ed way.' Thus, 
variables 15^ 16, and 18 represent a sort of "drill" 'in which 
the teacher asks, for' example , "What color is this apple, 
Susie?"; Susie answers, "red"; and the teacher responds, 
"That's right." ' , - 

The tables on variables 15, 16, and 18 show, as we 
might have predicted, Oregon and Kansas high on all three 
variabiles. 'There are some site effects, however, within 
these models 'on the first two variables: Tupelo in the 
Oregon model and Portageville in the ^Kansas model are far 
higher than other sites in .any model although the other sites 
i.n the Oregon and Kansas models are not at ail low. 

All three of these variables sliow a negative chcingc 
from fall to spring and we assume -that this finding supports 
our "rule" tha-t child^rcn needless rcv:al:d.ing , reminding, or 
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f 

Mean Score.s by Model on Classroom Obscrv^atipn 



Var. 15. Adult Direat Ouostioning of Child 





Fall 


Spring 


Far West ' 


'2. 782"^ 


' • 2.664~" 


U. of Ari^.,^ 


• 4.808 


4.454 


Bank St. 


3 248"" 

4 


3.435" 


U. of Oregon 


13. 359"^ 


10. 894'^'^ 


U. of Kansas 


8.415+ 


»7.606 + 


Hi/Scope 


7.0 94 


4.349 


U. of Florida 


5.208 


4. 26-6 

1 


EDC 


5.969 


6.104 


Pittsburgh 


8. 091 , 


4.569 


REC 


8.674 


7. 611 


Enablers 


3. 5^3- 


6.647 


Grand Mean 


6.125 


5.673 



Not e : In the above tabic + Indicates significantly high 
at .01 level (t value - 2.6 or over); -fH- indicates signi- 
ficantly high at .001 level (t value = 3.4 or over), 
indicates significantly low at .01 level; indicates sig- 
nificantly low at .001 level. 
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Mean ScQres by Model on Classrooin Observation 
Var. 16/ Child Response to Adult Direct Question 





. Fall ■ 


Spring 


Far West 


2. 092-; 


2.208 


U. of Ariz. 


3.097 


3.457" 


Bank St'. 


2.507" 


2.652" 


U. of Oregon 


12. 360*^"^ 


9. 712"^"'' 


U.^of Kansas 


^ 7. 308"*" ♦ 


r. 6.255+ 


Hi/SQopey 


5.842 


■ 3.191 


U. of Florida 


A one 

4.295 


O COT 

3.582 


EDC 


2.777 


4^019 


Pittsburgh 


6.488 


3.385 


REC 


6.802 


6.201 


Enablers 


2;097~~ 


. 4. 829 


Grand Mean 


',.774 


4-. 4 3 6 


Fall to Sp'iring 


change over all 


models ~. 



Note : In the above table + indicates significantly high at 
.01 level Ct value = 2.6 or over); ++ indicates significantly^ 
high at .001 level (t value = 3.4 or over). - indicates 
significantly low at .01 level; — indicates significantly 
low at .QOl level. - 



-245- ^ 

/ ' ■ "' 

drilling as they become more familiar with their glassroom 
env. ronment . 

Var 20: adult asking thought questions . 

Variah^le ' 20 shows a great deal of fall to spring 
change in several models. Far West, Oregon, Hi/Scope and" 
Enciblors all have lower scores in the spring. Only Pitts- 
burah and REC show more thought questioning in the spring. 
However, Arizona, Hi/Scope, EDC, and Pittsburgh remain 
above the overall mean on both observations , with Bank 
Stteet and Kansas significantly below the overall mean. 
/Thus, although model differences on this variable are con- 
fused by fall to spring change , it is clear that there are 
consistently more thought questions being asked of children 
in the Arizona, Hi/Scope , EDC, and Pittsburgli models than 
in the Bank Street and Kansas models. 

Var 27: child asking Questions 

• _ f 

We inclucje this variable mainly tio.show that the Oregon 
model, which is outstandingly high on adult cornmunicayfon focus: 
small group, and has high scores on the ques tion-re3ponse- 
feedback process, is significantly ^ low on 'child asking questions. 
This finding assumes more importance in connection with variable 
2.8 (chxld self-expression) , which is 'discussed in the next 
section. • • . 



4 
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3 



Mean Scores by Model on\ Classroom Observation 

_ 

Vat. 20. Adult Asking Thought Questions 





Fall 


Spring 


Far West 


1.538 


.0.697 


U. of Ariz. 


2.09,6 


1.764 


Bank St. 


0. 646- 


0.452- 


U. of Oregon 


3. 2 68++ 


0. 133 — 


U. of Kansas 


' 0.405-- 


0.397- 


Hi/Scope 


2. 689"*' 


1.671 


U. or Fioriaa 


■ 1 TZ. C Q 

i . b D O 


n Q Q "3 


EDC 


1. 753 


1.430 


•Pittsburgh 


1.723 


2.462'*""^ 


REC • 


0. 786 


1.67^ 


Enablers 


^ 2.442 


0. 344- 


Grand Mean 


1.726 


0.985 


•Fall to Spring 


change over all 


models 



Note : In the above table + indicates signi f icantly -high at 
.01 level (t value = 2.6 or over); ++ indicates • signi f icantly 
high at .001, level (t value = 3.4*or over). - indicates 
significantly low at .01 level; indicates significantly 
low at .001 level. 
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Mean Scores by Model cn Classroom Observation 





• 

v^xix i, lt J^\^^ a. ±1 1 g V SCI on S 






Fall ' 


Spring 


Far West 


4. 183+ 


1. 650 


U. of Ariz. 


2.727 
3.378 


1. 706 


Bank St. 


1. 851 


U. of Oregon 


0.775~- 


0.946- 


U. of Kansas 


2 . 3^r5 


1. 789 


Hi/Scope 


3. 224 


1. 54 6 


U. of Florida 


1. 333 


1. 954 


EDC 


1.279" 


1.668 


Pittsburgh . 


r.628 


2. 252 


REC 


2.620 


2.632 


Enabler^ 

\ 


3.553 


2.418 


Grand Mean 


2-. 620 


1. 800 


Fall to Spring 


change over 'all models ,. ~ . 




\ 





Note : In the above table t indicates significantly high' at 
.01 level (t value = 2.6 or over); ++ indicates significantly 
high at .001 level (t value ^ 3.4 or over). - indicates 
signifi.car^tly lev: at .01 level; ^indicates sigivi f icaivtly 
low at .001 level. 
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TABLE 35 

Third Sunimary • Table 



Adult Dir. Child Resp. 
Questning, to Adult ' 
of Child Dir. Quest. 



Adult 
Feedback 
to Child 
Response 



Adult Ask- 
ing Thought 
Questions 



Child 
"Asking 
Questions 



Fall ,Spr. Fall Spr . Fall Spr . Fall Spr . ' Fall Spr . 



Far West J 
U. of Arizona 
Bank St. 

U. of Oregon ++ 

U. of Kansas + 

Hi/Scope 

U. of Florida 

EDC \ 

Pittsburgh 

REC 

Enablers 



++ 



I^ote : In the above table + indicates significantly high at 
"TCn level (t value = 2.6 or over); indicatss si-gni f icant ly 

high at .001 level (t value = 3.4 or over).. - indicates 
significantly low at .01 lev/el; — indicates significantly 
low at .001 level. 
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Also, it is interesting that the Enabler model is 
consistently above the grand mean on this variable with 
small differences among its sites. We would not expect tire 
Enabler sites to be similar to each other, since there are 
no model speci f icati'ons , but all three sites show high scores 
on child asking questions. 

D, Child behavior (response) 

Var 39 : child positive affect toward adults 
28: child se 1 f -expression . 

Var 39: \child positive affect toward adults 

Variable 39 is the only one of seven variables concerned 
with affect that seems .to indicate any genuine differences 
among models, and even this variable shov;s significant site 
effects. The Hi/Scope mcjdel is consistently higher than other 
models, but their high t-value results from outstandingly- 
high scores for the Fort VJalton Beach ^ite. Pittsburgh is 
higher than the grand mean in the^ fall and is higher than any 
other model in the spring. 

It is puzzling that scores on child positive affect 
decreased over all models in the. spri^ng, especially since 
scores qn variable 28, (child seSrf-expression) show a sigVii- 
f leant increase. 

* ■ ■ 

'Var 28: child si. I i-oxpresis ion 

✓ 

Variable 28 is 'a goOd illustration of the difficulty 
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Mean ■.Sc ore s by Model on Classroom Obse r y a t i on 



Var. 39. Child -Posijtive Affect Toward Adults 





Fall. 




Spring 


Far West 


0.982 




•0.288 


U. of Ariz. 


0.129 




0.129 


Bank St. 


1.060 


■ 


0.5tf6 


U. of Oregon 


0 . 037 


- 


0.364 


U. of Kansas 


0.273 




0.149 


Hi/Seope 


, 2.418++ 




1.185++\ 


U. of Florida 


•0.352 




0.086 


EDC 

Pittsburgh 


' 0.181 
0.909 




0.694 - 
1.272 


REG 


0. 226 




0.14 8 


Enablers 


0.234 




0. 360 


Grand Mean 


^ 0.692 




0.478 



/ 



Note^^ In the above tal:)'le -f indicates significantly high at 
■ , or level (t value -= 2.6 or over); -f+ .4-ndicates :.igni f ican j ly 
high at .001 levgl (t yalue = 3,4 or over). r-indicates 
significantly low- at .01 level; — indicates significantly 
low at .001 level. . - 



I 
I 
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Mean Scores by Model on Classroom Observation 





Fall 


Spring 


Far West 


16 . 


,'8 8 5''" 


17.^23 


U. of Ariz. 


15, 


, 012 


17. 857 


Bank St. 


14 , 


,268 


17. 040 


U. of Ore(^.n ^ 


4 , 


, 501 — 


7. 332 


U.. of Kansas 


11, 


, 304 ■ 


10.895 


Hi/Scope 


8, 


,103 


12.497 


U. of Florida 


16, 


,739+- 


21. 2 39 


EDC 


■ 6, 


,533- 


13. 271 


Pittsburgh 


11. 


, 361 


14.802 


REC^ 


13, 


,260 


11. 941 


Enablers 


14. 


,356 


15 . 508 


Grand Mean 


12. 


, 008 


'l4.368 



Fall to Spring change over ali|models 



Note_^ In the above table -f indicates significantly high at 
.01 l6\el (t value = 2.6 or over); -f-f indicates significantly 
high at .001 level (t valine = 3.4 or*over). - indicates 
significantly lov; at .01 level; -~ indicates* significantly 
low at . 001 level . ■ 
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) 

involved in finding actual differences among models on some 
of our variables.' Child self-expression scores increased from fall 
,to spring significantly more than others. Hov;ever, v;e sec from the 
table that only the Oregon model is consistently low and 
that only the Florida model is consistently hiqh, 

The Oregon mode 1 was also low on the chi Id ask ing 
questions variable and thus, children in tiie Oregon model 
'apparently i.nitiate interactions less frequently than 
children in other Planned Variation models. 

There are large site effectfs on this variable within 
"'models other than the Oregon model, and so we can make no 
further as sumptions about differences among models on- child 
^ self-expression- 

MODgL SUiMMARIE S ' ■ . 

In this section we will" summarize the findings on 
our tv/enty-two variables by model. Also, v;e will attempt 
to relate thes.^ findings to model implementation. In other 
words, v;e will suggest v:hich differences among -models ^can 
be related to model theory. For example, it is obvious 
that the high scores on academic acti.vity for the Oregon 
and Kansas models are a result, of sponsor emphasis on teaching 
childro]'! specj. fie acadertuc skills. On the other han^, it 
is difficult to ujiders ta:id why, for 'examp'ie, REC is high on 
adult ackno\;lcdgoment: ar^d lov; on adult praise of children, ; • 



/ 
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TABLE 35 j 



Third wummary Table 
9 



) 





Child 
Self- 
Express ion 




Child Posi- 
tive Af feet 

Toward 

Adults 




Fall Spring 




Fa]. 1' Spring'; 


Far West 


J. 






U. of Ari7.ona 








Bank Street 








of Oregon 




• 


7 


of Kansas- 








Hi/Scope 








U. Of Florida 









EDC 

^Pittsburgh 
REC 

Enablers 



Note : In the aljovo table ^+ indicates s.lgni f icantly high at 
.01 levul (t value ^ 2,6 or over); -^--f indicates sigiii f icant]. 
high at ^.001 level (t value - 3.4 ^r over). - indicates 
signi f ican illy low at .01 'level; -~ indic£ito3 significantly 
low at .001 level. ' " 

/ 



I 

I 



I 
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J since we find no model specification which' states fthat 
adults should acknowledge children rather than praise them. 
I Thus, we will attempt to relate model theory to scores on 

t variables when some relationship seems plausible. 



The re are several problems with drawing conclus j.ons 

about model implementation on the' basis of the classroom. ' 

observation data. These data are derived from ain instrument 

which requires rapid qoding of readily observable behavior. 

While some -models (Oregon, Kansas, Pittsburgh and REC) 

specify in fairly explicit terms what activities should 

occur in classrooms and how adults ^ should interact with 

children, others emphasise "experiontes " for children. 

Our data cannot measure whether teachers are providing 

"sel f-rew,arding " experiences for children (a major as;^:ect 

; of the Far West model) , nor whether aciul'ts are "expanding 

childrenTs language" (a part of the Arizona model). Thus, 

« 

models vary in the doqree to v;hich 'implementation- can Idc 
checked oy classroom observation data. 

Further, scores on our variables re^flect frequency 
of behavior. We have no way of knowing^ 'i^or exainple, v;hether 
1^ Bank Street's score on independent child ; activity (which is 

^Signi f icantiy higher than othe-r models) -.is-, hi^h enough to 

' ^ ■ ^ ^ \ ' ■ V- 

, rate that mo d e 1 as h i cri i 1 y i mp 1 e jnc\n ted. ' Pi a nn e J V a r i d t i on 

sponsors- <^ not express thei'r thec^y in terms of contrast 

Ivjith other soonsQjfG; therefore, c^jon thouch -triere mav be • a 
great deal of independuu^t activity (as^ compared v;itii other 

• I . ■ ■ ■ • r ■ ■ 

ERJC 



A- 
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mpdels) in Bank Street, we cannot drav; a firm conclusion 
that Bank Street is well-implemented in that area. 

Wi th these 1 i mi t at i on s in mi nd , v;g ch os e a" f e w 
variables for each mode] which might be theoretically 
important • A table qf means and standard de vi ations or/ 
^hese theoretically important variables is includ^l in each 



model's^ summary. Site scores are shown,* since site 
variation within models is often larger than variation 
^ong models • We would expect a model to be high on the 
variables included in its table, except where "low" (in 
parentheses) appears after the variable, in which case wo 
would expect the mo.del to be low. ^ -n^ 

A quick -perusal of the tables of theoretically import:ant 
variables serves to emphasize the limitations of classroom 
observation data in judging wheth*er models arc wqll implemented 
The four in6dcls (Oregon, Kansas, IPI and REC) wliich specify 
certain types of overt- behavior and supply academic curricula 
to thei r teachers appear to be better imp? iicnted than tb.c 
other models . Although these models may indeed be well . 
im]plementecl;^the fad:t'that our variables are more relevant 
to t^ir model specifications than to the dther models , makes 
us 'wonder whether the other models would appear to be well- 
implemented if v;e used a different typd of observation- 
instrument. 



Enablers and Florida have no tables of theoretically important 
variables becaiu>:: ^hoy have no /..cciM s:-;oci f ication:; for clu:^:s- 
room- beliavior . 
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^ 0 

Thus, our data do determine which models are higher 

9 

than others on certain kinds' of readily observable behavior. 
Using this data to .make assumptions about one model's 
degree of implementation would be improper, since rodel 
theories differ from one anotl^r too muc^ to make any one 
"observation instrument relevant to all moXels. 



ERIC 



FAR. WEST 

The Far West model-is significantly higher th-.n the 
grand mean in both the fall and spring on only one of the 
twenty-two variables included in the previous discussion. 
That one 'is variable 23 (child self-learning with concrete 
objects). Although ' child self-learning is theoretically^ 
important in the Far West model, it is only the Duluth site 

9 

which is significantly high! Sic;ni f ican tly' low scores 
appear on praise and on the adult dirqct question-child 
response-adult feedback complex. Thus, the Far VJest rrio'del 
is not particularly distinctive on most of our -measures of 
classroom behavior. / 

The table of theoretically important variaibles shows 
'that the two sites iri the Far West model are substantially 
different irom each 'other . For example, Tacoma has high 
/ sco'reS* on adult communication focus : one child, v;hilG 

■ ■ - : \ . , 



-257- 



> 
O 



u 

p.' 



ii5 



cn 












CN 


o 


vD 




00 


LA 


00' 




00 


















I-H 


o 


\D 






00 






CN 


I-H 


I/) 


ii 


CN 




(N 


cn 




o 


00 










cn , 


cn 


O 




to 


7^ 


O 


O 


i-H 


o 


I-H 


I-H 


I-H 


O 


' o 

I-H 




to 




to 


to 




O 


rO 






































00 


i-H 


o 






o 


lO 


o 






CN 


I-H 


cn 














vD 








I-H 








a. 






O 


I-H 


II 


CN 




f-H 


O 


CN - 


O 




v£) 










LO 


o 




I-H 


x: 


O 


o 


f-H 


I-H 




f-i 




I-H 


CN 




to 


to 






c 


I-H 




















f-H 

















C^CO vD vDf-^Tj-oO-^OO^OLOrvl OO'-HCNtO-i-H 

•H \0 ri- CN LO O cn to O 0> Oi 0> 

(h Ii CN o *-r LO vD to v£)»^LO LO to c^j CN rj o o^o 















































o 


o 


f-H 


O 


o ^ 


o 


f-H 


O 




CN 


ro 


I-H 




' to 


O 


o 


to 
a> 
















































































CO 


o 




vD 


00 




I-H 


to 


to 


00 


O 


Oi 


00 




to 


xn 


♦to 




I-H 




00 


a> 


lO 




oo 


00 


00 


vD 


o 




to 


I-H 


o 


o * 


I-H 


vD 




I-H 


u 


I-H 


o 


vD 


i—f 




I-H 


I-H 


vD 






to 


to 


CN 


. o 






tJL. 


OJ 










































o 


o 




*-H 


o 


o 




<N 


o 

I-H 




OJ 


I-H 


to 


CN . 


o 


o 




bO 






































C 




vD 


00 


cn 




LO 


LO 


I-H 


O 


vD 




LO 


o 












•iH 






to 




00 


to 


to 


to 








00 


o 


vO 


I-H 


CO 


CO 






11 . 


»-H 


o 


I-H 




vD 


to 




vD 








vD 


to 


0 


o 


o 








































OJ 






o 


o 


I-H 


o 


O 


o 


f-H 


O 


I-H 


o 


to 


O 


to 


I-H 


o 


o 


B 






















I-H 
















O 






































o 






































OJ 








f-H 


LO 




vD 


r-4 


I-H 




o 


o 


I-H 


cn 




00 


o 


to 








00 




O 






LO 




cn 


to 




LO 


00 




a\ 


to 






I-H 


Ii 




o 


vD 


I-H 


LO 


I-H 


vD 


to 


o\ 


' 00 


I-H 






fN 








01 







































TO 

t4H 


2: 


o 


O 




O 


o 


O 


vD 


1— 


I-H 


I-H 


to 


o 




I-H 


o 


o 




W 
























k 














C 




vD 


o 






o 


to 






LO 


o 




I-H 


o 


»^ 












cn 


I-H 






CN 




vD 


vD 


to 






to 




LO 










Ii 




I-H 




to 


v£> 


to 


LO 


1- ■< 




00 


f-H 




CN ' 




o 


1 o 










































C ) 






o 


I-H 


O 


O 


O 


r-H 


O 




o ' 


to 


I-H 


LO 


to 


o 


o 






































*t 








































I-H 












































LO 


vD 


VD , 


. LO 


CN 


*— { 


LO 


Cvl 


LO 








CN 


LO 






Q 


I-H 










VD 


O 


CnI 




CO 


CO 


to 




00 


a» 


CO 








I-H 


n 


I-H 


1— < 




LO 


OO 


I-H 




LO 


oo 


CN 


LO 


o 






o 


o 



CTJ , . . 

<4H ' O O CN O 0 ,0 -H O LO »-H 1-^ I-H lO CN o o 









c 








C 




c 




C 
























* 






cd 










O 








o 




o 








o 




o 




t/1 


e 




e 




6 




6 




0 




B 









•H O 



CN Ph 

U O 
cd Jh 









































c: 
































o 




*-> 


o 
















:3 






















'2 
















o 






o 






O 




p: 






















;^ 






o 






o 




•H 




to 


o 












C 




Uh 












LH 


o 








a. 






5=: 






o 


















r-H 


'0 




o o 


<u 




I-H 


o 




I-H 










f-H 


c 




f-H 


c 


V f 


r-H 




o 


-a 


•H 




c= 




•H 


+-» 




o 






o 






o 




• H 








> 




i» 






(A 














-a 






x: 


















o 
























o 


























o 






o 




rj 










o 


-J 








•J 






.•J 














•J o 




cd 






r-H 










♦ H 










* 






i > 


^ J 


o 


r J 
















o 














o 


.. rj 




I-H 


c^; 


no 


Ch 




• H 




3 


'a 


to 


is 


rH 


to 




C; 


i/i 




O 

.fj u 




• H 














fci 


o 






rj 










r H 


H r: 










o 








o 


r* 




o 






O 






o 


>. o 










4-> 


> 


c:J 


> 


u 


5 










a 


•-H 




CO 





I 

r 



-258- 

I 

Duluth has high scores on independent child activity. 

Fall to spring changes are puzzling; Far'West's 

spring scores decrease significantly more than other models 

-I 

on twelve of the total fifty-one variables; spring scores 
do not increase significantly more than other models on 
any varialjple. 



UNIVERSITY OR ARIZONA 




ERIC' 



The Arizona model is significantly lower than the grand 
mean on adult praise of children^ and on adult feedback to 
child response. We see from the table that Arizona's scores 
on adult asking thought questions are consistently higher 
than the overall mean. Also, we ^an see that the two sites 
do not much differ from eacn other, except perhaps on adul.t 
positive- corrective feedback where Lafayette is higli. This 
model seems to typify the "average" P.V.J model, at least insc- 
far as our classroom observation variables measure classroom 
activity. 

BANK STPJt^ET ^ 



TJie Bank Street model differs from other models mainly 
on the amount of independent chij-'d activity. From the 
following tc'ible, we see that although the Wilmington site is 
much higher than Boulder or Elinira, all tlireo site;:, liavc 
scores near or: above tlio qrand mean. Related to their liiqh 
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scores on independent activity are low scores on adult 
acknowledgement, praise, and feedback. Strangely, the 
Wilmington site has lov; scores on child self-expression, com 
pared v/ith Boulder and Elmira. Thus, chi Id • sel f-expression 
seems to be negatively correlated wi t^Ijindependont activity 
in the Bank street model, 

fWe decided that Elmira 's fantastic score in the spring 
lid positive affect to other childre-n must be anomalous 
perhaps due to obsf^rver effects or unusual circumstances in 
the classroom. ' ' - 

UNIVERSITY OF OREGON * 

The Oregon model has • high s^cores on all variables that 
are related to acadeini'c activity. Although there is some 
site variation , .botl^^^'^tes a^e above the grarid mean on all 
academic activity variables, with the one exception that 
■ Tupelo is low on aide ' s part ici:^ ration in academic activities 
Oregonr is high on' adult comrt\unication focus: small group, 
on adult praise, and on the adult direct question-child 
response-adult feedback sequence. . . 

Low scores appear on child asking questions and child 
self-expression, r Since the Oregon model spcci fies, academic 
iristruction in small groups, it is clear that our veiriables' 
are me as ur i n g i mp o r t an t as pe ct ^ j f th i s mode 1 . 
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UNIVERSITY OF KAI^SAS 



The Kansas model , like the Oregon ir.odel , is high on* 
variables related to academic activity. In f^ct the 
Kansas model di f fers . f rom. Oregon in only three aspects; 
Kansas has more aides and fewer teachers participating 
in academic activities; the adult communication focus 

* 

is one child rather than small group; and the- Kansas . 

" ■ " ; - 

adults use much more positive corrective feedback e 

■'■V . " , 

Kansas is also low on independent' child acti'vity^. 
Obviously, our vai^iables measure the "behavior modi f icat ioA" 



process in soine del^^ail. ' . ' , 

.. . ' ' 

It is puzzling that the Portageville site ^is so much" 

f 

higher than Oraihi or Mounds on the adult direct question- 
child response-adult feedback seq,aence as Portagevi H e is 
not much higher on academic activi,ty. Perhaps Portageville 

teachers employ the direct question sequence in areas other 

> 

than academic activity. ^Except for this variation, the 
# three Kansas sites are quite similar to one another.* 



HI/SCOPE 

f' ' 

The Hi/Scope modfel is s J.gni f icantly high in both 
fal^l aiid spring on only one of our twen ty--t|j^o variables: 
child positive affect tc^P^rd adults. On adults asking 
thought quesuions, Ili/Scbyo is significantly high in tne fall 
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and above the grand mean in the spring. Hi/Scope is 

significantly low on several variables relating to academic , 

activities, / 

'^From the Hi/5>cope table, we see a very • mysterious .pattern 

The Fort Walton Beach site is quite di f f erent from Greeley 
( 

and Seattle in the fall on several variables but Fort Walton 

Beach changes radi cally ""f rom fal]. to spring. Since wo have 

both site variation and fall to spring change in this model, 

it is difficult to discover any consistent pattern ^among its 

scores. Perh^aos the Fort Walton Beach spring obse*rvation 

\ 

occurred under unusual circumstances. 

UNIVERSITY OF FLORIDA 

Because the Florida mtOdel does not specify any ^^^rti- 
cular classroom activity, but rather stresses training 
parents to teach their cfiildren by means of home visits, 
we have no tcible of theoretically important variables. As 
we might have predi cted , %vite variation is large. The 
Florida model is, however, signi f ican^jtly high on child self- 
expression. Perhaps the lack of sponsor pressure on adults 
in the classroom results in childz'en feeling more frec^ to 
express^ them.seives. s 

EDC • ' ' 

The KDC RiOdcl, like tii<^ arizona raodc 1 , soojns to be 
' a r o lui d t h e ci r a w d mo ci n o n i\ 1 n;os t all of o u r v a r i jiJj 1 c^. 3 . 5 1 1 c 
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A 



Variation is --large, as .v;g might have predicted since BDC 

emphasizes adaptation of the model to the particular needs 

of the site . EDC is one' of the Planned Variation models 

which stress " attitudes " and "providing for chi Idren assuming 
•- -v' , . * 

responsibility" rather than specific types of ^activities . 

f 

Children should be absorbed m theirVclassroom activity, 



and thus adults should provide "absorbing" materials and 
activities. Our variables cannot measure what is important • 
in the JfDC model; our variables can only measure immeaiately 
recoani zablc' kinds of interaations ^and activity, ^not whether 
those acti vi tic\s^ and interactions -indicate that children are 
involved or whether thev are beina creative. 



UNIVERSITY OF ,P ITTSBU R GII 

-- — — — ■ ^ — ■ — ■ — • 

The Pittsburgh model's spring scores indicate interest- 
ing classroom behavior; the model, is hj-gh on independent 
children in academic activities, adult in te ractj.ons with one 
or two children, and adult acknowledgement. Pittsburgh i-s 
also high on adults asking thought questions. Thus, Pittsburgh 
t«uachers do not supervise- acadontic act i vj.ty but rather acknow^- 
ledge children ' s v;ork and ask thought "questions ""of one or " 
tw o ch i 1 d re: n i n d i v i d u a 1 .1 V' . , 

^ Pi ttsbur oh * s sprinc: scores seem to bc^ consis tei:t wi th 
model ^theor'/ : the model eniphasizcs independent learning 
and t-j aclie r r:-:in force ;iA.-:iL of c^h. i Id lo arning • Of cours^.: , 
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therc is only one site in this model, 'and so we cannot 
compare scores across sites. However, there is strong ., 
evidence to suggest that in at least t>his one site-, ^ ^ 
Pittsburgh, made great strides tov/ard good model implementation 
from fall 1970 to>5pring 1971, ^ : 



REC 



REC is si,gT)ificantly high in both fall and spring on 
only one variciblo: adult acknov/lcdgeifient of children- 
From REC*s theory ^ we would expect higher scores' on 
i p e n d o n t/^'zzi v i t \' a n c 



independon^*^" a'C^i vi ty and on adult communicati.on focus: 
One child. In fact*"', from the T^C table we see Vihat 'REC- 
scores are above the grand mean, Iput^not high enough to be 
significant. 

Although REC, like Pittsburgh, was new to PlaiYned 
Variation in 1970-71, it did not change ^rom fall to spring. 
In fact;, REC changed le.-is than any other model. Thus, REC 
scores are consistently around the grand mean on almost all 
our variables. . 

The Enablor model has no prescribed classroom theory, . 
and thus, v;e could ncft select ' theorct^ically important vari- 
ables for tliis model,^ ' Because dach site is to be impleraented 
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according . to the community's needs arc desires, we v;ould 
anticipate more variation^ among the Enabler sites than in 
any other model. In fact, there is a great deal of site to 
site variation in this model, but not more than in the 
Hi/Scope, Far West, and Arizona models. 

Enabler sites', as a group, are significantly h'igh 
in the fall on adult' acknowledgement' and on children asking 
question^. All three sites are low on academic activity 
variables. The fact that Enabler sites arc not supposed 
to be alike perhaps indicates that the implementation 
of a model in two or more different sites is a complex and 
difficult. process . 

CONCLUSION 

'It is .apparent from the discussion of tl'ie classroom, 
observation data that interpretation of model difference^; is 
inhibited by 3e\'oral factors. First, the* research desi^jn 
is not balan.ced: some models have three sites others ^.o/ 
and still others only one. Further > the number of class- 
rooms observed differs from site to site (from one to 
four P.V, classrooms). Such an uiibalai^^^d desicin liuiits 
the statistical analysis of ^modol effects,* because sijic 
effects as well as classr6o]ii effects make moctol differeiKjes 
difficult to estaJ;] lisli . . ^ 



Further, our variables are limited to measiaros of . 
avert, evcii gross, behavior in the classroom. Those, models 
which emphasize behavior modification appear distinctive on 
our variables, but models which do not etnphasize particular 
methods of instruction do not appear distinctive. An 
instrument^ whi ch measured the context and purpose of adult, 
behavior miqht be more effective in revealing differences 
among models . , ' , 

In order to use clas..room observation data as, evidence 
of progress toward model implementation, we not on]!y>^^d 
a more sensitive observation i.nstrument, but also need more 

than two observations over the year. With only two obsorva- 

i ■■ ' 

tions, it is impossible to distinguish whethtir fall^ to 
spring differences are a result of actual change bve^r the 
year- '<^r merely a result of random day to day di f ferences ' in 
classroom activfties. -* , \ ^ 

Despite the above limitations, two s tatomeruts are possible 

1. We find strong evidence of differences anioncf models 
'on academic activity, independent child^a.gti vity , , 

and type of adult attenltion to chi^ldren. For exampJe, 
the Oregon and Kan^as/^ ass rooms appea:; vcrv different 
from Bank^treet and Pittsburgh classrooms. 

2. However, Fen: VJesl:, Tucson, Hi/Scope, Florida, EDC, 

REC and Enable r classrooms seem to be rather similar. 

Perhaps these classrooms actually are similar, but 
✓ ♦ 

tliG claGsroora cbsorvatien data docs not provide onougli 
inforraation about class, rooT,;.-, to cr-tablish tliat thcro 
arfi no ir.\portan-t diffcrcncon arp.ohu tliose mdcls.. 



i ^ 

I 

I 

* 

I 



I 



* Chapter 6 
CONCLUSIONS 




Summary Fir.clings : 

.nis report o>:|>lorod four if^sues: the clo^rGO to which 
niodcVs aro } irtplomc:]^- ; tho variation .in factors which may 
• affect imp] c^p^oTlt ci t- J r.n ; th^ rcjation bo twccn' those fxactors 
and level?; of i raplrM ymiI a t i on ; and the variations amon(?f models 
in claS5;roon1 act I'vi ti es , 

^ We addressed the question of how well the models are 

iniplem.ented through an analysis of the sponsors* ratings of 

teachers. We find that: 

* ^ 

'1. All models on the average are luOfietately well 
implemented . ' * 

2. There are no- significant differences among models.. 

3. Tnere is large variation . among classes v;ithin 
sites in levels bf i.mplementati.on as ty<rH>->^s 

' sigrii f ican t dif f erenees ctmong sites v;ithin models . 

4 

V J 

4. The' levels of implementation for^ second year sites 
or for second year teachers are not significantly 
higher tliaii f o!r - f irs t year , and not all models show 
imprcnenicnt over the year. ' ^ 

' ♦ - • . . ' • 

Frc^m. t!;ese findings , v.-e concludg^-^hat the implemicntation of 

models is iiot as si^mple as liad been ori.ginally suppc^sed; v>e 



ERIC 




cannot say thatc model s can be delivered to all classes, 
n-'ir can we say that the experimental treatments are fully 
implemonte'd . Moreover, these f'indings rpise tv/o Additional 
questions': v;hy is there variation in level of implementa-' 
tion, land if models are, not being fully implem^Mited , v;hich'' 
parts are being implemented? ' - ■ ' 

In trying to explain, the ^'ariation, we explored a 
variety of factors v/hich might affect impl cementation . We 
propose that implementation is an intea'acti^'-' process depend- 
ing not only on spc>nsor input, but also on staff input and 
react.ior ; and on the operational context of the staff. Because 
i:hero ::re rroblems with the data available for f^ach of these 
areas, we must be cautious in our interprct(A -ions . Some 
findings, however, are clear: ^ 

1. There are . signi f icant differences amiong ir.odels and 
■ . . among sites v/ithin m.odels in tl^e ty].^cs and amounts 

of trainina received bv the teachers . 

2. There are significant site and model differences in 
the training given Id^ people other than the sponsor. 

3. There are significant differences among sites and 
•* - models in the characteristics of the teachers. 

4. There arc no significant differences amonc} models 
in the contexts in which implementation takes 
place, but thcr do appear to be differences among 
sites ..within mode 
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The finding of model and site differences on a largo nurhcr 

of vari'ables support? the afgumerlt-...that impl^enltation , . 

is complex, and that sorr'ie of these vaj^^iTables may i bo useful 

in explaininq vari ation in level 'of irovl einoht-?.tinn . 7- ' -the 

same time, the discussion of factors wliich af f ect jimplcrnent a-^' 

tion reveals that-mpro study is needed ItDOCause ■ there are ' 

many important dimensions of implem.on/a ti on for which the present 

data are inadeouate, or for v/hich there are no data at all-.' 

The discussion of factors which influence :i mplortHMi tri i oi'i 

also suggests, two -areas in which traditional exoeriliient a] , 

- ■ - -J 

requirements " aro not met. The findings of systematic diffc^n^- 

ences among sites, and more importan'tly ■ar:^orxg model's, in staff 

characteristics and in^ the training given by people o.ther 

than .-the sponsor indicate »that inputs other than the models 

may be confounding f actors These factory should be , considered, 

therTr^hen .drawing^ conclusions about m.odol effectiveness. • 

After identifying the fact^g^ which might affect iiitple- 

mentation, we entered! some of the vajjiables in a serie^^ of 

regression equatiqns an attem]:^t tc) exii-^lain the- /sponsors ' ^4 

ratings of levels. o.f imp] ementation . We find, thaw? 

1. As we exf>ect^d from the earlier analyses of variance, 
models, do not explain a sixjni^f i'cant amounb of the vari a- 

tion in levels of imv-^loment ation " . 

2. Site-to--site variation . accounts for* approximately 

■30% of the total variance-. 
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3. We can explain 98"v of the varialnce airong sit r, in 

I 
1 

levels of implementation v:ith ree vari ables : 
sponsor's jucigm.ents of intra-staff friction, 
rapport betv/een the staff and the administration/ 
and adequacy of the physical plant .indoors. 

4. Wi thin-site var.iab].es add only 10?; to the explained 
variance after the sit^ variation i si controlled* 

This failure to explain within-site variation underlines the 
contention that we do not have adequate knowledge about 
the process of implementation. The most impressive finding 
is -^hat we can explain essentially all of the variance among 
sites in levels of implem.entaticn'. Two interpretations 
of this finding are plausible. It is possible that sponsors, 
in rating the teachers^ are simply equating pleasant sites 
.with high levels of implementation: site differences may_ 
reflect only differences in pleasantness, and may be un- 
related to actual differences in pe>-for.mance within the model 
This interpretation is supported by the fact that jadgnients 
on the same three-variables by consultants are not related 
to levels of implementation. It is also possible that pleas- 
ant sites actual] V do have higher levels of implementation tJ 
do unpleasant sites. This may be true either because a 
pleasant site is a pierequisite for implementation or because 
the staff in a site are so enthusiastic about a laodel that 
they v;ork v:ell togot;.c;r in order to ir:ple:v;cnt it. Alt:i^.ao;i 
it is difficult to make a definitive statc::nent about the,. 
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cauKal relation betv/een the site: factors and levels of 

/ - 

implementation, this finding is' both important and i.ntor-- 
esting. The relationship' Of site atmosphere and model' 
implementation should 'be explored further. 

Tho Classr60Tn Observation data ar\ used to discover 'v >V7 
modojs diffo]' in tornis of C"]. aps rrvoii^ acVrv^ty and v/hothcr 
tlioso di ffcM-onc-C'S ^ arc j n tori^^TX- tal'.>].o i,n tcrp\s of jrodo] 'theory 
There arc throe iTtain f indi nc^s ' 



1. Orcqon and Kansary appear to be particularly well 
implor.cntod , sho\\^ no higli scores on the variables 
which reflect thoir model proscripti ens . 

2. Bank Street and Pittsburah seem to .'"e well imple- 



i^entod in the areas of independent cl^ld activity/' 
and adult attention .to children, ^ 
3. Far West, Arizona, Hioh Scope, Florida, EDC, RF-C , 
and Enal.^ler classrecnis c^o not appear to be siqni- 
ficantlv different froin one another on most of the. 

"at ■ . 

variables. Ho^'^cver, eaehi or these models is siqni- 
ficantlv '^Icfh or low on at least one classroom oli'Se 
vation variable. y \ , 

Thus , the Clas.^room Observa-tion chapter reports that 
some iriod.^^s CcHi be ro'-jdily differentiated jn several miCasures 
vdiiie otiK-r^iiiodels seem to bo ^ rather -sirviilar to ore another*. 
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Implications for Future Studios 



It appears that traditional experimental requij^ement s 



itiohal experimental reqiu^e 



that^the treatments are well-specified and f'^ly 



implemented — are not met in. Planned Variation.^ The 
contention that treatments were not well-siDOci f led \^?«js 
discuF^sed in Chapter 1: v,^e do not hcive a clear picture 
of what a class using each mo3el wou.-u look like ; ^cause 
.not all Planned Variation m.odels can be fully described in 
behavioral terms. The finding tfiat the models are not' 
.fully implemented 'is summarized a.bove , Such factors 
complicate an experiment because they make the interpre- 
;t:ation of effects difficult, ' * 

It is important, then, to ask why the treatment require 
men.ts were not met in Planned Variation, in order to deter- 
mine v;hethei: the deviations wi] 1 re-occur in future studies 



If we coulA conclude that thev resulted from. inadequate 
planning, then v;e could expect to have \>tudieg^ .with well-^ 
defined and fully impleiacnted treatnfents by simply taking 
moi*e ca ^ tc) plan well. Our evidence , 'b9th quantitative 
and anecdotal, however, guygests that deviations do not 
simply result from poo'r planning but stem from the nature 
xpf tKe treatments and of the impleraentat^or/ prOeess , 

The nature of some of the Planned Vair^ition niodels 
indicates that they will never be well-^specif led in a con- 
ventional sense because th';'ir philosophy ru:is covuiter 



-282- 



to close prescription . Instead they set out oono rn 1 

principles and encouraqe toacher.s to carry them out 

in a manner v;hich bent , fits their sty] e /and the needi- 

of their students, Tiie l^DC and Bank Strec^t i^'oc^eJs are 

wall-knov:n exar^ples of this type of proqrant. 'i'iiir, is 

not to say that n^odel-v. of this kind catinot l.)0 carofulJv - 

defined we slroncjly rccoii^p/'^^nd tliat tjus Ik- dcfUc^ wlicvre 

— - _ ^ 

it has rot oeen , already . The Jevv-^l of d< i pt j qi-^. , 

-hovevcr/ iiviy not be ovc-rt classroora beha\'lor wliicrli ro- 

mains constant in all situations, 

^The nature of the models tor:p$her wit/i th<- nature of 

the .intplementation oroc*ss alscj indicates t/iat treatments 

' ^ • . ^ * r 1 r 

will not ^OG fully implcmentc^d^ in tiie sense of., findinq all 

class'es to be replications c the ideal' models. In p^rt 

this ^assertion C9mes from the di^cuasion ajjove: if a model 

can take a variety ^of • fgrm.s ,^iien identical replicaticn 

snould _not. be- expected. In part, it ^s a function of the 



We ex- 



interactive nature of 'the .i^^^>4e*meittatioh process 

^ ^ ' \ ' . ■ - y ^ 

pect some variation in al-1 models eveii^in tiio^e* models . 
which do not p-.couraoe variation ."- ocicause riinplementation 
depends not only on tlie model and the sponsor input , but 
on site factors , such as the attitudes and s];ills of thc^ ^ 
staff and the organization of th^^ Head Start center , 'v.'hj.ch 
are beyond the co.ntrol of liie researciiors . 'x'^ius it appears 
that the deviations i.n. 'P lanned Vari at ion stcmi from the nature 
of the situaticjn and Ih^ reCore, will tend to occur in any stuciy 
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- J 

involving complex treatments v/hich depend on local f 

people for implc^'ir.entation . V7| would argue that given 

the inevitab.i li ty of^ the "messiness" of such studies , it 

-should be taken inta accrn^nt in planning other studies. 

Moreover, it*" should be recognized that the messiness of' 

» 

a fj.eld study, a.^ide from the . experiment al p^^Qblems it % 
presents, .is not reallv" a noqat ive phenom.enon • In^ sense, 
it is preferable to laboratory rese'arcli because 3,t haVs greater 
ecological' validity : treei^-ments are tested under conditions 
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which af)proximate those they will face' in the real wor. 
V:. should accept the vagaries of a study 'like HSPV, then, 
not only because they are inevitable,- but because we can 
learn a great deal from them. Two broad recommendations 
can be made for future .studio-^. 

1 ^ Alterna tive^ standards ' for a valid experiment; mi ust 
be develoged. .This involves both defining fiujl implementa- 
tion realisticallv and taking differences in implementation 
into apcount in analyzing ^treatiiient- effectiveness . The 
important issue in redefining fulj implementation is -allowing 

s^)me variation among -classes: v;e.must accomodate the fact 

%^ 

that a m.odel may be. adapted, by the sp.onsor and the teaching 
staff, to ti"ie uuj.quo situation in a site, or even to a 
class v;ithin a site, and to chciiigcs in a situation over 
time-: Even i^' those joodels which do not emphasize adapta-- 
tion, identical rcplic:;t:i.cr» should not'-^bc expected. v;e need, 
'hov.'cvor, to srjt liiViits on the variation: v;e m.ust determine 



/ 



♦ 
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hov; much variation can exisiis^ v/hi Ic still consi cl.' ring a cln.ss 

to bo a fully i mplor^.ontocl program. Vic must bo - able to docido 

at v/hi ch point the variati on is so great that t'ul J imp] crwo.n- 

tation becomes partial implemenrat J on , and at wl/ich point 

parti.al imp] emionta t .i on i.^ecome:-'. notiiinq no cvidon^'-c^ of tin- 

'no specific operational definition of full implementation. 

Th^s is not to say ^ that s|:Xinsors cannot identify a well- 

implcK^ented c^ass, only that .they have not systematized, 

* \ * 

or at least communicated, .the^ir criteria. 

Provision for taking differences in impl^entation 
into account in the outc.om'e analyses is important 'because 
we'would expect, even with a i/edef inition pf full imple- 
mentation, that ^cme cla'^es will be less than fully 
implej:nented a tea'cher- r»ay only use portions. cf a model, 
or only use it part of the time. 'Taking differences into 
account can be accompli^'hed either by setting a crituri'on 
level or by usijig the extent of: implementation as a covariate 
In the case o'f the fornier, we v;ould decide v;iiat level of 




^implementation represented an acceptable level of treatmcMi-t, 
and only analyze effects in classes Which exceed that level. 

would essei'itialiy be liiaking a siniple dist ii^ict ion between 
treatment and no treat^i^ent. The "^ritical Issue in nUikiiKj 
this simple distinction .is deciding wheru the eul'-QfC point 
should tall. h di:- dvcintage to this app?.Ooch i:^ th.'it it 
potentially saciificc-:. a gr^^ut dv^^l of u...t.;. Vl-^ ^ 
approac^i, using iraplon.enta tiori i.'Z n , ccvj iv: i.c: or a'-' 
^ . \ 
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independent variable in a regression equation to control 
for the variation .in effects resulting from differences 
in iraplementation , does make use of all available data. 
Several other issues^ however, must, be con^^idered. First, 
it is important tha'U.-th^ scaJ^s used to* measure the extent 
of implemen tati on bq gan chorea such that st anda rds for 
judgmonts , particula^jy the criteria for ful^ implementation 
and for no i.miplem.eni^ ti on , are made explicit. This, was 
not done with the sponsor ratings of teachers described 
earlier, and, as a result, the conclusions drawn from them 
can only be tentative. Second, the prok^lem of comparing 
part ia]^ implementation should be resolved. Within a 
single model, one class whichXis, say^ 60% implemented 



may not look- like ancther\vh^h is 60% implemented 

because di-fferent par\s of the mod*! may<be present in each. 

The problem is to determine when these classes can be con- 

- \ 

sidered to be acceptable, r^^plicatioris of., a single treatment. 
Criteria should be establishc^^i to make this determination, 

■ ■ . . \ , ^ , ' 

but to. date, it has 'not beeu done. Compar?uSon across 

models is more difficult and probably should not be attempted 

- ^ . O i \- - 

because models have dif/erent components and ave at diffej^ent 
levels of operationali^iation . \^ 

|fetting all these staridards is a nece'^sary basis for 
deterruiiiincj v;hethcr the treatiaent as implem6;;:ted is an 

acceptable exaiuple of the treatraerit we set out\to test . 

\ 

Although the sta!:idaraf: iv.ay !:)0 diffe?:cnt than in a \convon 'cional 
experiment, tliey 'aucit still be met. If the standard.^ 

\ 

•cannot be establis?;ed fpr example, if lii.ats cannot 
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set on the variation allov/od, at least to + he extent of 

being able to specify V7hat differentiates a ir.odol class 
from non-model classes, and to specify v7hat the model 

classes have in com>mon — then, \ we do not have a valid 
experiment. . 

r 

2. Xh^.Jll^''^-^'lL^^^'-^l^^^ itsol ^ , an 

im.por t^"^ nt tor>-ir: for study,- Within c\ traditional 
GXpor.i mental framework, the focus of the research is on 
the effect of the traatments. In' this context, the study 

c 

of implementation is seen primarily as a simple check- on 

{I, 

the presence of the treatments. When we cannot assume 
that all classes will reach full implementation, it is 
eauallv important to examine the question of whether 
and in what form a prcQram will be implem.ented , and tc^ 
identify tlae factors- which determine the process. 

In this report, a great deal of speculation and son^c 
analysv:-s with data .originally collected for other purposes 
have been done on individual factors which might affect 
implem.entation . What is needed, .lowever, is a study of the 
iinplemei'itat 3 .">n process: a study designed t^ ask what , 
effects, sponsors ' involventent , training efforts, and ttaff 
organ^iz'ation had; to ask v;hat characteristics of th. f local 
staff, of the Head Start organi^at^ion, of cho coiTtmunlty 
in v/hich the cetnor is located affect implementation; and 
to develop a theory "of hov; these factors are interrelated 
— i.e. , to dovelo}^ a tricory of pioccss rather: tli^n' of 
individual ractors. This might be done either by stud\jing 
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natural variation in delivery systeir.s or by experimenting 
with pre-specif led strategics. Thi? type of study goes 
beyond the issue of experimoirta 1 validity to the investi-- 
gat ion of conceptions of plarn^d change . The exi sting 
literature on innovations,^ on ilntervention theory, on org 
i/ations, and on decision maki^icj should be useful in 
further \>qnccptuaj.ixing tho^ issues. 
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